[CMake] Skip(ped) test with CTest?

[David Cole]

On Fri, Jun 8, 2012 at 11:39 AM, Christoph Grüninger <
christoph.grueninger at iws.uni-stuttgart.de> wrote:

> Hi David,
> thanks for your answer and the bug tracker link. I don't have any new
> arguments besides that the arguments and use cases stated in the bug
> tracker apply to our use case, too.
>
> I see it like a poor man's code coverage. Once I forgot to enable MPI
> support in autotools and it reported 47 tests passed, 1 skipped. I cannot
> memorize the absolute number of tests for every program but the mentioned
> skipped tests made me investigate which test was skipped. After
> reconfiguring all 48 tests passed and I was sure, my MPI related change did
> no harm. If the skipped test was reported as passed, I actually would
> falsely assume all tests passed.
>
> As already stated in the bug tracker, I would report skipped tests as "not
> run" and leave CDash as it is. Only CTest needs changes, so no "can of
> worms" to be opened.
>
> Thanks for your answer,
> Christoph
>
>
>  This has been requested before:
>>
>> http://public.kitware.com/Bug/**view.php?id=8466<http://public.kitware.com/Bug/view.php?id=8466>
>>
>> It's in the backlog, would require changes to both ctest and cdash, and
>> I'm not enthusiastic about it. Perhaps I just need convincing that it's
>> a good idea. For now, there are no plans to implement anything like
>> this, and I would recommend that you use a wrapper script approach,
>> printing out "skipped" and returning success when you don't actually run
>> your test.
>>
>>
>> HTH,
>> David
>>
>>
> --
> People who are more than casually interested in computers should have
> at least some idea of what the underlying hardware is like. Otherwise
> the programs they write will be pretty weird.         -- Donald Knuth
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/**
> opensource/opensource.html<http://www.kitware.com/opensource/opensource.html>
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/**CMake_FAQ<http://www.cmake.org/Wiki/CMake_FAQ>
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/**listinfo/cmake<http://www.cmake.org/mailman/listinfo/cmake>
>


There is a test property called REQUIRED_FILES, which is a list of files
that must exist in order to run a test. According to the ctest source code,
we will return a status of "Not run" if one of those files does not exist.

So one way to achieve your goal would be to list the test via add_test
always, but then require a file for the test to run, and make sure that
file does not exist before ctest_test is run.

So it seems like you should be able to achieve your goal using that if you
would like. As long as you're ok with the "Not run" status.


HTH,
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20120608/83885b2b/attachment.htm>