<br><br>On Sunday, January 8, 2012, Fraser Hutchison <<a href="mailto:fraser.hutchison@googlemail.com">fraser.hutchison@googlemail.com</a>> wrote:<br>> On 07/01/2012 14:41, David Cole wrote:<br>><br>> On Fri, Jan 6, 2012 at 6:47 PM, David Doria <<a href="mailto:daviddoria@gmail.com">daviddoria@gmail.com</a>> wrote:<br>
><br>> On Fri, Jan 6, 2012 at 5:54 PM, Jean-Christophe Fillion-Robin<br>> <<a href="mailto:jchris.fillionr@kitware.com">jchris.fillionr@kitware.com</a>> wrote:<br>><br>> Hi David,<br>><br>> Not too long ago I was browsing the project of a friend who worked on BTK<br>
> (The toolkit used by Mokka, a motion kinematic & kinetic analyser) [1].<br>><br>> I noticed that he is using cxxtest [2] along with ctest. It seems it could<br>> to address the use case you are describing.<br>
><br>> See Example integration [3] and cxxtest source [4]<br>><br>> Hth<br>> Jc<br>><br>> [1] <a href="https://b-tk.googlecode.com/svn/web/mokka/index.html">https://b-tk.googlecode.com/svn/web/mokka/index.html</a><br>
> [2] <a href="http://cxxtest.tigris.org/">http://cxxtest.tigris.org/</a><br>> [3]<br>> <a href="https://code.google.com/p/b-tk/source/browse/BTK/trunk/Testing/Code/C3DFileReaderTest.h">https://code.google.com/p/b-tk/source/browse/BTK/trunk/Testing/Code/C3DFileReaderTest.h</a><br>
> [4]<br>> <a href="https://code.google.com/p/b-tk/source/browse/#svn%2FBTK%2Ftrunk%2FUtilities%2FCxxTest%2Fcxxtest">https://code.google.com/p/b-tk/source/browse/#svn%2FBTK%2Ftrunk%2FUtilities%2FCxxTest%2Fcxxtest</a><br>
><br>> Thanks Jean-Christophe. So I guess the short answer is "ctest can't do this".<br>><br>> It seems they have made significant modifications to CxxTest so that<br>> the bin/cxxtestgen that ships with CxxTest is no longer required<br>
> (they've replaced it with C macros). I like the BTK way better :)<br>><br>> I guess I'm not sure what the advantage of using CTest along with<br>> something like CxxTest is. If you build an executable using CMake,<br>
> then why not just run ./MyTests from the terminal instead of making an<br>> interface so that you can call 'ctest'. CTest will report a single<br>> pass or fail, because as far as it is concerned there is only one<br>
> test, right? The relevant file is<br>> <a href="https://code.google.com/p/b-tk/source/browse/BTK/trunk/Testing/Code/CMakeLists.txt">https://code.google.com/p/b-tk/source/browse/BTK/trunk/Testing/Code/CMakeLists.txt</a><br>
> that only has one add_test call.<br>><br>> Any thoughts?<br>><br>> David<br>> --<br>><br>> Powered by <a href="http://www.kitware.com">www.kitware.com</a><br>><br>> Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br>
><br>> Please keep messages on-topic and check the CMake FAQ at: <a href="http://www.cmake.org/Wiki/CMake_FAQ">http://www.cmake.org/Wiki/CMake_FAQ</a><br>><br>> Follow this link to subscribe/unsubscribe:<br>> <a href="http://www.cmake.org/mailman/listinfo/cmake">http://www.cmake.org/mailman/listinfo/cmake</a><br>
><br>> Sounds like you want to use GTest / GoogleTest, much like SimpleITK does.<br>><br>> See the macro "ADD_GOOGLE_TESTS" in the SimpleITK project, in the file<br>> SimpleITK/Testing/Unit/CMakeLists.txt.<br>
><br>> The macro parses the actual test cxx source files, and constructs<br>> "filtered" add_test calls, such that:<br>><br>> - if a cxx file defines 5 tests, add_test is called 5 times, each test<br>
> run via add_test runs exactly one of your tests from the cxx file.<br>><br>> There are other projects that are also doing similar things with<br>> gtest. The fairly new open chemistry project MolCore also uses gtest.<br>
> That project explicitly lists its tests in the CMakeLists file such<br>> that you'd have to update the CMakeLists.txt file whenever you add a<br>> test case to an existing cxx test source file.<br>><br>> You are correct: ctest does nothing to support this explicitly. (Aside<br>
> from cmake providing a FindGTest.cmake module...) An add_test call<br>> tells ctest "run this one test, with this one command line." It's up<br>> to you to write those tests and tell us about them.<br>
><br>> But there are fairly easy ways to do what you want, and still drive<br>> the tests, individually, with ctest.<br>><br>><br>> HTH,<br>> David C.<br>> --<br>><br>> Powered by <a href="http://www.kitware.com">www.kitware.com</a><br>
><br>> Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a><br>><br>> Please keep messages on-topic and check the CMake FAQ at: <a href="http://www.cmake.org/Wiki/CMake_FAQ">http://www.cmake.org/Wiki/CMake_FAQ</a><br>
><br>> Follow this link to subscribe/unsubscribe:<br>> <a href="http://www.cmake.org/mailman/listinfo/cmake">http://www.cmake.org/mailman/listinfo/cmake</a><br>><br>> We found a few issues using gtest in the way you describe David:<br>
><br>> Parsing C++ code isn't CMake's thing, so it becomes increasingly difficult to support anything other than very simple gtest macros (like TEST_P, TYPED_TEST, TYPED_TEST_P). We do handle these macros now, but I'm sure it's pretty fragile, and we don't even attempt to handle things like C++ namespace rules within test files!<br>
> gtest provides the ability to set up (potentially costly) test environments which persist throughout execution of all tests, rather than on a per-test basis. Using this, and then having CTest invoke each test individually totally defeats the purpose of this feature.<br>
> Commenting out a test using block comments leaves the test name unchanged, so CTest still tries to run it.<br>> If a gtest exe is run with a filter which doesn't match any test names, it runs 0 tests and returns 0 which CTest interprets as success. This could happen if 3 above were the case. We've got a policy of using the following in main() to avoid reporting success:<br>
><br>> int result(RUN_ALL_TESTS());<br>> int test_count(testing::UnitTest::GetInstance()->test_to_run_count());<br>> return (test_count == 0) ? -1 : result;<br>><br>> gtest exes can be run with the flag --gtest_list_tests which outputs all test names in that exe. It would be great to use that to generate the list of filters needed, but that suffers from needing the exe to exist before CMake is run. So if a new test case is added, it won't be run by CTest until CMake is re-run.<br>
><br>> Having said all that, we continue to like gtest and are fairly used to the limitations mentioned above. Whatever problems we've hit, there's always been a CTest way to get round the issue.<br>><br>
> Cheers,<br>><br>> Fraser.<br>><br>><br><br>Interesting. Thx for chiming in.<br><br>Let us know if you have any suggestions or good ideas for how we could improve things. Maybe a test property that identifies an exe as a gtest exe, and then runs it to figure out the list of tests to run.<br>
<br><br>David<br>