[CMake] Code Coverage & Bullseye

Thu Sep 18 08:00:33 EDT 2014

As David suggested I did following experiment.

I created two Test projects.

*Project1: *

*set(CTEST_SOURCE_DIRECTORY /Users/rajekumar/work/test-codes/TestUnix)*

*set(CTEST_BINARY_DIRECTORY /Users/rajekumar/work/test-codes/TestUnix/out)*

*set(CTEST_CMAKE_GENERATOR "Unix Makefiles")*

*set(CTEST_BUILD_COMMAND "make")*
*Project2:*

*set(CTEST_SOURCE_DIRECTORY /Users/rajekumar/work/test-codes/TestMac)*

*set(CTEST_BINARY_DIRECTORY /Users/rajekumar/work/test-codes/TestMac/out)*

*set(CTEST_CMAKE_GENERATOR "Xcode")*

*set(CTEST_BUILD_COMMAND "xcodebuild")*

Project1 is able to do the code coverage.Here is the log snippet.

*SetCTestConfiguration:CoverageCommand:/usr/bin/gcov*

* Add coverage exclude regular expressions.*

* target directory list
[/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/TargetDirectories.txt]*

*Performing coverage*

* COVFILE environment variable not found, not running  bullseye*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/Continuous.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousBuild.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousConfigure.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousCoverage.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousMemCheck.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousStart.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousSubmit.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousTest.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ContinuousUpdate.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/Experimental.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalBuild.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalConfigure.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalCoverage.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalMemCheck.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalStart.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalSubmit.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalTest.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/ExperimentalUpdate.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/Nightly.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyBuild.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyConfigure.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyCoverage.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyMemCheck.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyMemoryCheck.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyStart.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlySubmit.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyTest.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/CMakeFiles/NightlyUpdate.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestUnix/out/test/CMakeFiles/Sample.dir*

*   Processing coverage (each . represents one file):*

*    ."/usr/bin/gcov"  -o
"/Users/rajekumar/work/test-codes/TestUnix/out/test/CMakeFiles/Sample.dir"
"/Users/rajekumar/work/test-codes/TestUnix/out/test/CMakeFiles/Sample.dir/main.c.gcda"*

*File '/Users/rajekumar/work/test-codes/TestUnix/test/main.c'*

*Lines executed:100.00% of 2*

In Project 2  ctest was unable to find the .gcda. It was in

/Users/rajekumar/work/test-codes/TestMac/out/test/Project.build/Debug/Sample.build/Objects-normal/x86_64/

and ctest was searching in following directories(from logs):

*globbing for coverage in:
/Users/rajekumar/work/test-codes/TestMac/out/CMakeFiles/ALL_BUILD.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestMac/out/CMakeFiles/Continuous.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestMac/out/CMakeFiles/Experimental.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestMac/out/CMakeFiles/Nightly.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestMac/out/CMakeFiles/NightlyMemoryCheck.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestMac/out/CMakeFiles/ZERO_CHECK.dir*

*   globbing for coverage in:
/Users/rajekumar/work/test-codes/TestMac/out/test/CMakeFiles/Sample.dir*

So when I gave the following command manually it worked.

*RAJEKUMAR-MBP:TestMac rajekumar$ /usr/bin/gcov -o
/Users/rajekumar/work/test-codes/TestMac/out/test/Project.build/Debug/Sample.build/Objects-normal/x86_64/
/Users/rajekumar/work/test-codes/TestMac/out/test/Project.build/Debug/Sample.build/Objects-normal/x86_64/main.gcda
*

*File '/Users/rajekumar/work/test-codes/TestMac/test/main.c'*

*Lines executed:100.00% of 2*

*/Users/rajekumar/work/test-codes/TestMac/test/main.c:creating
'main.c.gcov'*

So it seems that if generator is Xcode then the .gcda files are created in
/Users/rajekumar/work/test-codes/TestMac/out/test/Project.build while the
ctest try to find it  in
/Users/rajekumar/work/test-codes/TestMac/out/test/CMakeFiles

Is there any way to make ctest to search in some additional directories  ??

Rajeev

On Thu, Sep 18, 2014 at 3:36 PM, David Cole <dlrdave at aol.com> wrote:

> As I said in my previous response, I don't know anybody who is using Xcode
> on the Mac, and also successfully producing a gcov coverage build.
>
> If I were you, I would:
>
> (1) use gcov on a Linux build of your software for coverage measurement,
> using the "Unix Makefiles" generator and the right gcov flags
>  or
> (2) buy Bullseye for the Mac, as NoRulez suggested his company did because
> they couldn't get a reliable coverage build using gcov on the Mac
>
> This is not a ctest problem. This is a "gcov doesn't work right on the
> Mac" problem.
>
> If somebody would like to claim gcov does work right on the Mac, I would
> love to see the proof. If anybody is doing it successfully, please do share
> your scripts and a link to the dashboard results you get using it. (And any
> hints we might need to replicate it ourselves...)
>
>
> Thanks,
> David C.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20140918/179c1b92/attachment.html>