[CMake] CTEST_CUSTOM_PRE_TEST and child process disassociation

[Eric Noulard]

i am interested in using CMake for client/server testing
and asked once for advice, you can read the
answer kindly provided on ML here:

http://www.cmake.org/pipermail/cmake/2006-October/011453.html

I keep thinking on how to "implement" extensible client/server testing
using CTest.

I didn't know about CTEST_CUSTOM_PRE_TEST
and seems interesting.

Nevertheless It seems to me that CTest has not been designed
for client/server and/or distributed testing.
It may be out the scope of CTest too (ANY CTest designer may answer this?)

I dot have much time to go further on this now but I'm definitely interesting
in this "distributed testing" issues.

My guess is that if you want to have a real good support
you should have some kind of way to control the execution
of all your clients AND server apps.

The tools will need the kind of features you may find in Tcl Expect
http://expect.nist.gov/.

I would rather go for another scripting language like Python
may be using pexpect (http://pexpect.sourceforge.net/)
or twisted (http://twistedmatrix.com/trac/).

I would be pleased if I can "plug" such scripted control
directly into CTest.

The question are:

a) is it a CTest objective to "support" distributed testing
    May be we just want CTest to support "identified external scripting"

b) If yes, how to do it the good way.


Any comment on this subject would be highly appreciate.

2006/11/10, Eric BOIX <eboix at ens-lyon.fr>:
>         Dear cmake users,
>
> Consider a package that offers some client/server functionalities. In such
> a context some regression tests (as defined with ADD_TEST(...) ) concern
> the server and some others tests concern the client. Typically, I thus need
> to launch a server at the beginning of ctest, then to run each individual
> client test, and then to cleanly kill the server (oh well, in theory :).
>
> For this, I use CTEST_CUSTOM_PRE_TEST which I configured to be a shell script
> (I know this is not portable but, for the time being, I don't know any
> better way). This typical CUSTOM_PRE_TEST is of the form:
>    #!/bin/bash
>    the_server_to_test > log &
>    echo Done
>
> The problem is that ctest seems to wait for the disassociation of the
> child process (in this case the server I wish to test) before proceeding with
> each atomic test. This can be easily checked by configuring
> CTEST_CUSTOM_PRE_TEST with the following simple script:
>    #!/bin/bash
>    sleep 30 >log &
> for which ctest will wait for 30 seconds i.e. the end of the child process
> before proceeding with the test suite per se. In despair, and just to
> make sure, I also tried
>    sleep 30 < /dev/null >log 2>&1 &
> and in drunken despair I even tried
>    nohup sleep 30 < /dev/null >log 2>&1 &
> but ctest behavior seems to always be the same...
>
> Interestingly enough when I launch those shell scripts manually from
> the command line (as opposed to launching them from within ctest) I then
> get my hands back on the invoking shell while the server (on in the
> above examples the sleep command) keeps on running...
>
> My questions is thus: when using CTEST_CUSTOM_PRE_TEST how can one force
> ctest not to wait for child process disassociation ?
>
>         Yours,
>         Eric Boix.
>
> _______________________________________________
> CMake mailing list
> CMake at cmake.org
> http://www.cmake.org/mailman/listinfo/cmake
>


-- 
Erk