[cmake-developers] Unit testing CMake modules

[Sam Spilsbury]

<self-promotion>

A little while ago I wrote a framework to do just that:
https://github.com/polysquare/cmake-unit

I haven't maintained it in a few years, but it works exactly the way
you would expect it to (the syntax is a bit different though).

</self-promotion>

On Wed, May 30, 2018 at 12:36 AM, Wouter Klouwen
<wouter.klouwen at youview.com> wrote:
> Hi all,
>
> We have a rather large amount of CMake code (4k+ lines) in various
> modules/functions. They contain the common logic for many of our
> projects. This makes it quite an important base for everything our team
> does.
>
> Rather shamefully at present these modules are rather poorly tested. I'd
> like to improve this, but the current way of testing CMake code
> is typically to run a trimmed project, and to verify whether certain
> invocations produce a certain output or file hierarchy.
> This involves a bit of infrastructure and can be a bit cumbersome
> maintain, to diagnose when tests fail, and it requires a separate run of
> the tests.
> The overall cumbersomeness of the setup in turn discourages in our team,
> including myself, from adding tests.
>
> I'd like a more integrated approach that makes running at least some
> basic tests part of the build progress, and a more direct way of
> reporting failures.
>
> In other programming environments, testing often involves some kind of
> mocking environment, and CMake helpfully allows the overriding of CMake
> built in functions, though this is typically discouraged.
>
> In my ideal world it would be possible to save the state of the current
> function set up, then call a function with a certain given number of
> parameters, and expect a certain sequence of events, such as functions
> to be called, environment variables to be set, etc.
> Something akin to CMakePushCheckState, except for built in functions.
>
> Then a module could provide for some functions that would set up
> expectations and verify them, within a run of cmake, or possibly some
> other commands could be added, to give some syntactic glossy coat to it.
>
> As it wouldn't actually trigger any of the expensive generating
> functions, it would be lightweight, quick to run and give pretty direct
> errors in terms of failed expectations, reducing debug time.
>
> If it was done with CMake commands, I might imagine it to look something
> like:
>
> function(foobar)
>    # function to test, does something non trivial
>    if ("FOO" IN_LIST ARGV)
>       install(FILES foo DESTINATION foo_dir)
>    else("BAR" IN_LIST ARGV)
>       message(FATAL_ERROR "Some error")
>    else("BAZ" IN_LIST ARGV)
>       set(BAZ True PARENT_SCOPE)
>    endif()
> endfunction()
>
> test(foobar)
>    expect(WITH "FOO" CALL install FILES foo DESTINATION foo_dir)
>    expect(WITH "BAR" CALL message FATAL_ERROR "Some error")
>    expect(WITH "BAZ" ENVIRONMENT BAZ True)
> endtest(foobar)
>
> What do people think? Is this crazy? Is there a quicker way to get
> somewhere close? Should I put some effort into making this into an
> actual proposal/working code?
>
> Thanks in advance,
>     W
>
>
> This transmission contains information that may be confidential and contain personal views which are not necessarily those of YouView TV Ltd. YouView TV Ltd (Co No:7308805) is a limited liability company registered in England and Wales with its registered address at YouView TV Ltd, 3rd Floor, 10 Lower Thames Street, London, EC3R 6YT. For details see our web site at http://www.youview.com
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> https://cmake.org/mailman/listinfo/cmake-developers



-- 
Sam Spilsbury
pgp.mit.edu: 0xb8a90fb599bae9c2