[cmake-developers] [ANNOUNCE] cmake-unit - A new unit testing framework for CMake

[Sam Spilsbury]

Hi all,

By way of introduction you might remember me as a maintainer of Compiz
between 2009 and 2012, which is the window and compositing manager
still used by Ubuntu today. We used CMake quite heavily in Compiz and
so I've learned a great deal about it.

Over the last two years I've been working on a new unit testing
framework for CMake. I started this project because I've been writing
some other fairly complex CMake modules and I kept making mistakes
which would take me hours to debug weeks down the line. I would have
announced the project's existence a lot earlier, though I became
hesitant of doing so when I rewrote it earlier last year. Now that I
see there are discussions about potentially changing CMake's scripting
language, I figure it is probably better later than never.

The project is, rather unimaginatively, called cmake-unit. The key
difference between this and the RunCMake framework that's used by
CMake internally is that cmake-unit more closely resembles an xUnit
architecture. It has support for a generic assert_that function and
matchers, test-case autodiscovery, coverage report generation and
allows you to keep all your tests together in a single file. The only
downside so far is that it runs a fair bit slower than RunCMake does.
Presumably this is because under the hood it uses a lot of
indirection. On the upside, I have been finding optimisations to help
speed things up, so hopefully performance won't be a problem in the
future.

So far I have it hosted as a third party module on GitHub[0] and it
can be added as a dependency to biicode projects[1]. I also wrote a
blog post which gives a little bit more detail as to how it all works
[2]. The project has reached a relatively mature state by this point
and I have used it in several of my own projects successfully. I'd
like to reach out to a broader audience and get some more feedback to
keep things moving forward.

One thing I'm not sure about is whether this is an appropriate kind of
project to integrate into the CMake mainline. If that were to occur,
it'd have the upside of bringing the functionality to many more users.
The immediate problems I foresee are dealing with how this framework
would interact with RunCMake and also dealing with some of the hacks
that cmake-unit uses internally. In particular, calling arbitrary
functions is done by abusing variable_watch, which works, but isn't
exactly ideal.

Feedback and patches definitely appreciated,

Best Regards,

Sam Spilsbury.

[0] http://github.com/polysquare/cmake-unit
[1] http://biicode.com/smspillaz/cmake-unit
[2] https://smspillaz.wordpress.com/2016/01/25/a-unit-testing-framework-for-cmake/

-- 
Sam Spilsbury