[CMake] Custom target for running tests as part of the build

[Steve Lorimer]

Thanks Matt, that works great!

As a note for if someone wants to repeat this in the future...

One of the things I initially tried was bash stdout and stderr redirection
(using &>)

&>${TEST_NAME}.output


This doesn't work. I guess it causes the test to run in the background? It
also results in the test.passed sentinel always being created.

To get both stdout and stderr I had to redirect stdout to file and stderr
to stdout

>${TEST_NAME}.output 2>&1



On 23 August 2016 at 12:14, Matthew Keeler <mkeeler at tenable.com> wrote:

> One thing I see immediately is this line:
>
> $<TARGET_FILE:${TEST_NAME}> > ${TEST_NAME}.output || cat
> ${TEST_NAME}.output
>
> Unless ${TEST_NAME}.output doesn’t exist cat is going to have a successful
> exit status and cause the whole COMMAND to be successful. Something like
> “|| (cat ${TEST_NAME}.output && false)” should work in bash. When the test
> passes everything in the parens won’t be executed but when it does it will
> still force a bad exit status.
> For a cross platform solution I would create a cmake script to do the
> grunt work of executing the test.
>
> COMMAND ${CMAKE_COMMAND} -DTEST_EXEC=<TARGET_FILE:${TEST_NAME}>
> -DTEST_NAME=${TEST_NAME} -P test-runner.cmake
>
> Then in the script it could do something like the following:
>
> execute_process(COMMAND ${TEST_EXEC} OUTPUT_FILE ${TEST_NAME}.out
> ERROR_FILE ${TEST_NAME}.out RESULT_VARIABLE TEST_SUCCESS)
> if (NOT TEST_SUCCESS EQUAL 0)
>     execute_process(COMMAND cat ${TEST_NAME}.out)
>     message(FATAL_ERROR “Execution of ${TEST_EXEC} exited with a failure
> status”)
> endif()
>
> execute_process(COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed)
>
>
> Depending on how much you need the cross platform solution the ugliness of
> the extra indirection may not be desirable for you.
>
> --
> Matt Keeler
>
>
> On August 23, 2016 at 11:20:48, Steve Lorimer (steve.lorimer at gmail.com)
> wrote:
>
> We have several unit tests which we would like to be run as part of our
> build process.
>
> To achieve this I have a helper script which creates a custom command that
> runs the test, and if successful, creates a file "test_name.passed".
>
> I then add a custom target "test_name.run" which depends on
> "test_name.passed".
>
> The idea is that if "test_name.passed" doesn't exist or is older than
> "test_name", the custom command will be run.
>
> Builds will continue to run the custom command until the test passes.
> Subsequent builds won't call the custom command, so the test won't be run
> if it doesn't need to.
>
> Here is the script:
>
>     # create test.passed command which runs the test and creates a
> sentinel file if it passes
>     add_custom_command(
>         OUTPUT  ${TEST_NAME}.passed
>         COMMAND $<TARGET_FILE:${TEST_NAME}>
>         COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
>         DEPENDS ${TEST_NAME}
>         )
>
>     # create test.run module which depends on test.passed
>     add_custom_target(${TEST_NAME}.run
>         ALL
>         DEPENDS ${TEST_NAME}.passed
>         )
>
> The problem is that our tests often log a tonne of information to stdout,
> and it makes for a very noisy build.
>
> I'm trying to now capture stdout to a file, and only in the event of a
> failure, display the test output.
>
> My first attempt was to try a Bash shell scripting syntax - capture stdout
> into a file and when the exit status is an error, cat the file.
>
>     add_custom_command(
>         OUTPUT  ${TEST_NAME}.passed
>         COMMAND $<TARGET_FILE:${TEST_NAME}> > ${TEST_NAME}.output || cat
> ${TEST_NAME}.output
>         COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
>         DEPENDS ${TEST_NAME}
>         )
>
> This doesn't work, as even if the test fails I am getting the sentinal
> "test_name.passed" file created, which means the next time I try to build
> it thinks the test passed.
>
> Is there a way to achieve what I want? Bonus points for a cross-platform
> method, but if it has to be Linux only, then so be it.
>
> Thanks in advance
> Steve
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20160823/7281d327/attachment-0001.html>