[cmake-developers] [ANNOUNCE] cmake-unit - A new unit testing framework for CMake

[Brad King]

On 01/25/2016 11:01 AM, Sam Spilsbury wrote:
> Over the last two years I've been working on a new unit testing
> framework for CMake. I started this project because I've been writing
> some other fairly complex CMake modules and I kept making mistakes
> which would take me hours to debug weeks down the line.
> 
> The project is, rather unimaginatively, called cmake-unit. The key
> difference between this and the RunCMake framework that's used by
> CMake internally is that cmake-unit more closely resembles an xUnit
> architecture. It has support for a generic assert_that function and
> matchers, test-case autodiscovery, coverage report generation and
> allows you to keep all your tests together in a single file. The only
> downside so far is that it runs a fair bit slower than RunCMake does.

Nice.  I think if something like this were to be provided by upstream
CMake it would be better to implement it directly in C++, at least for
the test execution and output matching parts.  That would be much faster
than a pure CMake language implementation and also would not need hacks
to execute CMake language functions.

One of the weaknesses of RunCMake infrastructure is that the CMake regex
engine is not meant for large expressions.  For example, the number of
()-groups is limited to 10 or so.  It is also not as expressive as
modern regex implementations offer.  There has been discussion about
replacing the regex engine previously:

 https://cmake.org/pipermail/cmake-developers/2011-November/014249.html

but there are major compatibility challenges:

 https://cmake.org/pipermail/cmake-developers/2011-November/014376.html

OTOH in a brand new interface we could use an alternate regex engine
from the beginning.

-Brad