[cmake-developers] ctest -j 32 hanging

[Thomas Sondergaard]

Hi,

I have observed twice now within the last month that ctest with -j is 
hanging. It seems to not notice that a test has ended and it is 
furthermore not reacting to the timeout given on the command line.

The following output is from a jenkins log. The job invokes ctest like this:

ctest --output-on-failure --timeout 10800 -C Debug -j %NUMBER_OF_PROCESSORS%

where number of processors is 32. Note that it should have stopped the 
job well before jenkins didafter 300 minutes as 10800 seconds is only 
180 minutes and the compilation that is done before running the tests 
takes less than 15 minutes.

I logged into the machine while ctest was hanging and test job #3 
mitest-test was not running any longer.

> Test project F:/jenkins/workspace/personal-ts-7.0-all-tests-3/arch/win32/build
>       Start  1: micppunit-test
>       Start  2: evbase-test
>       Start  3: mitest-test
>       Start  4: tinymath-test
>       Start  5: mistral-test
>       Start  6: usg-test
>       Start  7: mint-test
>       Start  8: charls-test
>       Start  9: vdbclient-test
>       Start 10: qtutility-test
>       Start 11: yargnits-test
>       Start 12: trident-test
>       Start 13: sturgeon-test
>       Start 14: camutility-test
>       Start 15: general2d-test
>       Start 16: hl7tools-test
>       Start 17: session-manager-test
>       Start 18: libwfm-test
>       Start 19: common-tests
>  1/19 Test  #1: micppunit-test ...................   Passed    1.20 sec
>  2/19 Test  #4: tinymath-test ....................   Passed    1.89 sec
>  3/19 Test #16: hl7tools-test ....................   Passed    4.95 sec
>  4/19 Test  #7: mint-test ........................   Passed    7.71 sec
>  5/19 Test  #8: charls-test ......................   Passed    7.66 sec
>  6/19 Test #18: libwfm-test ......................   Passed   14.21 sec
>  7/19 Test  #6: usg-test .........................   Passed   18.10 sec
>  8/19 Test #17: session-manager-test .............   Passed   18.35 sec
>  9/19 Test #13: sturgeon-test ....................   Passed   19.66 sec
> 10/19 Test #15: general2d-test ...................   Passed   19.70 sec
> 11/19 Test #10: qtutility-test ...................   Passed   24.13 sec
> 12/19 Test #14: camutility-test ..................   Passed   33.26 sec
> 13/19 Test #12: trident-test .....................   Passed   33.76 sec
> 14/19 Test #11: yargnits-test ....................   Passed   40.25 sec
> 15/19 Test  #9: vdbclient-test ...................   Passed   50.64 sec
> 16/19 Test #19: common-tests .....................   Passed   61.04 sec
> 17/19 Test  #5: mistral-test .....................   Passed   77.47 sec
> 18/19 Test  #2: evbase-test ......................   Passed  138.26 sec
>
> Build timed out (after 300 minutes).

I searched the cmake bug tracker for problems with ctest hanging, but 
did not find any. Anyone aware of problems such as this?

I grabbed a dump of the hanging ctest and I will see if I can get 
anything out of it.

Best regards,
Thomas Sondergaard