[cmake-developers] Testing Groups

[Matthew Woehlke]

On 2013-06-10 15:49, Alexander Neundorf wrote:
> On Monday, June 10, 2013 09:19:15 AM Andreas Schneider wrote:
>> Hi,
>>
>> I'm currently working on some libraries [1] which allows you advanced
>> testing of daemons. Especially ones which require a network in testing.
>> However for using this I need some features in CMake. This should be a
>> discussion about these features. Maybe others have ideas and comments.
>>
>> What I would like to have is a feature to group tests. By default everything
>> is in a standard test group. But you can define a special one:
>>
>> add_test(NAME <name> [CONFIGURATIONS [Debug|Release|...]]
>>           [GROUP <test_group>]
>>           [WORKING_DIRECTORY dir]
>>           COMMAND <command> [arg1 [arg2 ...]])
>>
>>
>> For a test group I need a way to define a setup/teardown command. Some tests
>> might require a special setup on the machine to be able to run correctly.
>> Like setting up a fake network.
>>
>> set_test_group_properties(PROPERTIES
>>      SETUP <command> [arg1 [arg2 ...]]
>>      TEARDOWN <command> [arg1 [arg2 ...]]
>>      ENVIRONMENT <env>)
>>
>> The setup function is called before the first test of the group is executed
>> and the teardown functions after the last test finished. The environment
>> should be obvious.
>>
>>
>> Please comment.
>
> Why not simply set up and tear down that network for every test ?

I would guess that might work poorly for parallel tests :-). (Plus it's 
inefficient. Depending on the time to do set-up / tear-down versus 
granularity (i.e. number) of tests, I could imagine a reasonable 
scenario where doing it per-test increases the test time by at least one 
order of magnitude.)

-- 
Matthew