[CMake] CTest outside of CMake

[Tim Gallagher]

Hi,

Sorry I wasn't clearer. That section was from a script that we call with ctest. The entire section that runs things looks like

# Load our custom settings                                                                                                
ctest_read_custom_files("${CTEST_REPO_DIRECTORY}")
ctest_start("Nightly")
ctest_update()
foreach(TESTCASE ${LESLIE_AVAILABLE_TESTCASES})
  set(CTEST_CONFIGURE_COMMAND "./setup.py -t ${TESTCASE} ${LESLIE_CONFIGURE_DICT}")
  ctest_configure(BUILD "${CTEST_REPO_DIRECTORY}" APPEND)
  set(CTEST_BUILD_COMMAND "./setup.py -t ${TESTCASE} ${LESLIE_BUILD_DICT}")
  ctest_build(BUILD "${CTEST_REPO_DIRECTORY}" APPEND)
endforeach()
ctest_test(BUILD "${CTEST_REPO_DIRECTORY}")
ctest_submit()

And we call ctest with 

ctest -S CTestScript.cmake 

My question is, if I only want to run a few tests, ie only run tests 2, 3, 4:

ctest -S CTestScript.cmake -I 2,4

This only run the ctest_test() on tests 2, 3, and 4, but I don't know how to determine inside the script which tests will be run, so it configures/builds all of the available tests since I have that loop. 

So, is there a variable that gets set or some way to determine which tests are called from the command line inside the script in the event something like -I or -R flags are used? Is there a variable that contains the list of tests that CTest will run?

Hope that's clearer. Thanks,

Tim

----- Original Message -----
From: "Eric Noulard" <eric.noulard at gmail.com>
To: gtg085x at mail.gatech.edu
Cc: cmake at cmake.org
Sent: Monday, October 17, 2011 3:06:48 AM
Subject: Re: [CMake] CTest outside of CMake

2011/10/17 Tim Gallagher <tim.gallagher at gatech.edu>:
> Hi,
>
> We have our project set up to run a series of test cases, but we are treating it as if we are not using CMake. I found a tutorial online on how to set it up, and we have it running just fine when we want to run all of the tests. But, I haven't figured out how to do the configure/build steps for selected tests only.
>
> For example, each test has it's own configure command and build command. If I do
>
> ctest -I 2,2
>
> then it configures and builds all the tests, but will only run the second test.
>
> The driver script contains a section like:
>
> foreach(TESTCASE ${LESLIE_AVAILABLE_TESTCASES})
>  set(CTEST_CONFIGURE_COMMAND "./setup.py -t ${TESTCASE} ${LESLIE_CONFIGURE_DICT}")
>  ctest_configure(BUILD "${CTEST_REPO_DIRECTORY}" APPEND)
>  set(CTEST_BUILD_COMMAND "./setup.py -t ${TESTCASE} ${LESLIE_BUILD_DICT}")
>  ctest_build(BUILD "${CTEST_REPO_DIRECTORY}" APPEND)
> endforeach()
>
> and the LESLIE_AVAILABLE_TESTCASES is a list of test cases names that match the add_test() test names. Is it possible when running CTest with -I or -R (or even without -I or -R) to figure out which tests it selects so we can build LESLIE_AVAILABLE_TESTCASES from that?

I'm not sure to understand your whole testing process flow,
Is the script above part of a ctest -S script?

Filtering some tests may be done with test NAMES or LABELS
see EXCLUDE / INCLUDE or
EXCLUDE_LABEL / INCLUDE_LABEL
 arguments of ctest_test (ctest --help-command ctest_test)

However you does not seem to use ctest_test ?
How do you run your ctest command (which arguments/options are you using)?

Concerning labels
see http://www.kitware.com/blog/home/post/11 and reference therein.



-- 
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org