[cmake-developers] Testing Groups

[Andreas Schneider]

On Monday 10 June 2013 21:49:41 Alexander Neundorf wrote:
> On Monday, June 10, 2013 09:19:15 AM Andreas Schneider wrote:
> > Hi,
> > 
> > I'm currently working on some libraries [1] which allows you advanced
> > testing of daemons. Especially ones which require a network in testing.
> > However for using this I need some features in CMake. This should be a
> > discussion about these features. Maybe others have ideas and comments.
> > 
> > What I would like to have is a feature to group tests. By default
> > everything is in a standard test group. But you can define a special one:
> > 
> > add_test(NAME <name> [CONFIGURATIONS [Debug|Release|...]]
> > 
> >          [GROUP <test_group>]
> >          [WORKING_DIRECTORY dir]
> >          COMMAND <command> [arg1 [arg2 ...]])
> > 
> > For a test group I need a way to define a setup/teardown command. Some
> > tests might require a special setup on the machine to be able to run
> > correctly. Like setting up a fake network.
> > 
> > set_test_group_properties(PROPERTIES
> > 
> >     SETUP <command> [arg1 [arg2 ...]]
> >     TEARDOWN <command> [arg1 [arg2 ...]]
> >     ENVIRONMENT <env>)
> > 
> > The setup function is called before the first test of the group is
> > executed
> > and the teardown functions after the last test finished. The environment
> > should be obvious.
> > 
> > 
> > Please comment.
> 
> Why not simply set up and tear down that network for every test ?

Cause it is simply a waste of time and resources!

What if I need a special VM with qemu for the test. Starting it and shutting 
it down for each test takes probably minutes. It makes more sense to set it 
up, run a bunch of tests and then shut it down.

If I need to start a daemon which is a reference implementation and I need 
just to test aspects of connection details, it will never taint the 
environment and there is no need to stop it.


	-- andreas

-- 
Andreas Schneider                   GPG-ID: F33E3FC6
www.cryptomilk.org                asn at cryptomilk.org