[CMake] CTest tests time out with CDash submission

Thu Jan 3 15:03:46 EST 2013

Why not just run "make Experimental" instead?

If your tests already run the checkers that you want, then just running the
tests is sufficient. You don't need to run make ExperimentalMemCheck if
your checkers run as part of your test suite...

Running the MemCheck runs all tests via the memory checker.

On Thu, Jan 3, 2013 at 2:47 PM, Christopher Mullins <
christopher.mullins at kitware.com> wrote:

> As a followup, I think I may have found part of the problem.
> When I run $ ctest -N -VV -D ExperimentalMemCheck, I see test #29 listed
> as follows:
>
> 29: MemCheck command: /usr/bin/valgrind "-q" "--tool=memcheck"
> "--leak-check=yes" "--show-reachable=yes" "--workaround-gcc296-bugs=yes"
> "--num-callers=50" "/home/crmullin/cppcheck/cppcheck/cppcheck" "-q" "--xml"
> "--suppressions"
> "/home/crmullin/TubeTK/Utilities/Cppcheck/TubeTKCppcheck.supp"
> "--enable=style" "/home/crmullin/TubeTK/Applications"
> "/home/crmullin/TubeTK/Base" "/home/crmullin/TubeTK/Utilities/Testing"
>   Memory Check  #29: CppcheckStyle
> Memory check command: /usr/bin/valgrind "-q" "--tool=memcheck"
> "--leak-check=yes" "--show-reachable=yes" "--workaround-gcc296-bugs=yes"
> "--num-callers=50"
>
> Then if I run $ ctest -N -VV -R "CppcheckStyle", I see this same test
> listed as follows:
>
> 30: Test command: /home/crmullin/cppcheck/cppcheck/cppcheck "-q" "--xml"
> "--suppressions"
> "/home/crmullin/TubeTK/Utilities/Cppcheck/TubeTKCppcheck.supp"
> "--enable=style" "/home/crmullin/TubeTK/Applications"
> "/home/crmullin/TubeTK/Base" "/home/crmullin/TubeTK/Utilities/Testing"
>   Test #30: CppcheckStyle
>
> The first method of invoking the test is running, but it runs incredibly
> slowly, because it is primarily running valgrind, and not cppcheck!  The
> second method is actually running cppcheck correctly, and thus it is
> running more quickly.  Does anyone know how to instruct CTest to use the
> correct executable?  I am having the same issue with the Cppcheck
> integration in CalaTk, it appears to be maybe the same issue (although it's
> hard to tell because Cppcheck is turned off by default in CalaTK, as we can
> see on the CalaTK dashboards<http://open.cdash.org/index.php?project=calaTK>
> ).
>
> Thanks for any help you can provide!
>
>
> On Wed, Jan 2, 2013 at 3:57 PM, Christopher Mullins <
> christopher.mullins at kitware.com> wrote:
>
>> Hello CMake community,
>>
>> I am attempting to incorporate Cppcheck into the code analysis section of
>> the TubeTK dashboard.  It's static code analysis, distinct from dynamic
>> code analysis tools like Valgrind, but I'm simply incorporating the checks
>> for Style, Performance, Portability, Information, and UnusedFunctions as
>> additional tests in the same category.  Basically I'm trying to mimic what
>> has been accomplished in the CalaTK project.<http://www.calatk.org/gitweb?p=calatk.git;a=blob;f=Code/Testing/CMakeLists.txt;h=2e900af70d5c6353b7dae38b5ebdb3282cbc4750;hb=HEAD#l1031>
>>
>> The issue I'm having is that the tests now run as such:
>> TreCime ~/Projects/tubetk-cppcheck/TubeTK-Build $ time ctest -R "Cppcheck"
>> Test project /home/crmullin/Projects/tubetk-cppcheck/TubeTK-Build
>>     Start 30: CppcheckStyle
>> 1/5 Test #30: CppcheckStyle ....................***Failed  Error regular
>> expression found in output. Regex=[error file=] 77.12 sec
>>     Start 31: CppcheckPerformance
>> 2/5 Test #31: CppcheckPerformance ..............***Failed  Error regular
>> expression found in output. Regex=[error file=] 74.77 sec
>>     Start 32: CppcheckPortability
>> 3/5 Test #32: CppcheckPortability ..............***Failed  Error regular
>> expression found in output. Regex=[error file=] 75.78 sec
>>     Start 33: CppcheckInformation
>> 4/5 Test #33: CppcheckInformation ..............***Failed  Error regular
>> expression found in output. Regex=[error file=] 75.37 sec
>>     Start 34: CppcheckUnusedFunction
>> 5/5 Test #34: CppcheckUnusedFunction ...........***Failed  Error regular
>> expression found in output. Regex=[error file=] 75.02 sec
>>
>> 0% tests passed, 5 tests failed out of 5
>>
>> Total Test time (real) = 378.13 sec
>>
>> The following tests FAILED:
>>  30 - CppcheckStyle (Failed)
>>  31 - CppcheckPerformance (Failed)
>>  32 - CppcheckPortability (Failed)
>>  33 - CppcheckInformation (Failed)
>>  34 - CppcheckUnusedFunction (Failed)
>> Errors while running CTest
>>
>> real 6m18.135s
>> user 6m17.444s
>> sys 0m0.332s
>>
>>
>> This is great.  It takes only a few minutes to complete, and fails when
>> it should.  The same tests are *timing out* when I run $ make
>> ExperimentalMemcheck:
>> Start  27: ImageCompareCommand-Test6_fetchData
>>  27/618 MemCheck  #27: ImageCompareCommand-Test6_fetchData
>> ...............................................................................
>>   Passed    3.50 sec
>>         Start  28: ImageCompareCommand-Test6
>>  28/618 MemCheck  #28: ImageCompareCommand-Test6
>> .........................................................................................
>>   Passed    3.08 sec
>>         Start  29: CppcheckStyle
>>  29/618 MemCheck  #29: CppcheckStyle
>> .....................................................................................................***Timeout
>> 1500.04 sec
>>         Start  30: CppcheckPerformance
>>  30/618 MemCheck  #30: CppcheckPerformance
>> ...............................................................................................***Timeout
>> 1500.04 sec
>>         Start  31: CppcheckPortability
>>  31/618 MemCheck  #31: CppcheckPortability
>> ...............................................................................................***Timeout
>> 1500.03 sec
>>         Start  32: CppcheckInformation
>>  32/618 MemCheck  #32: CppcheckInformation
>> ...............................................................................................***Timeout
>> 1500.04 sec
>>
>> During this time the process appears to be CPU-bound, so I don't think
>> there is some blocking issue that I'm not seeing keeping the test idle.
>>  Has anyone else encountered this problem?  Is this a CTest issue at all?
>>
>> Thanks for all your help,
>> --
>> Christopher Mullins
>> R&D Engineer
>> Kitware Inc.,
>> 919.869.8871
>>
>
>
>
> --
> Christopher Mullins
> R&D Engineer
> Kitware Inc.,
> 919.869.8871
>
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20130103/d392856a/attachment-0001.htm>