[CMake] building tests

Wed Jul 14 10:36:20 EDT 2010

On 07/14/2010 06:40 AM, Paul Harris wrote:
> On 11 July 2010 20:36, Michael Hertling <mhertling at online.de> wrote:
> 
>> On 07/10/2010 06:54 PM, Paul Harris wrote:
>>> On 9 July 2010 22:39, Michael Wild <themiwi at gmail.com> wrote:
>>>
>>>>
>>>> On 9. Jul, 2010, at 15:48 , Michael Hertling wrote:
>>>>
>>>>> On 07/08/2010 09:47 AM, Paul Harris wrote:
>>>>>> On 8 July 2010 15:31, Michael Wild <themiwi at gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> On 8. Jul, 2010, at 7:25 , Paul Harris wrote:
>>>>>>>
>>>>>>>> On 8 July 2010 12:56, Michael Wild <themiwi at gmail.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 8. Jul, 2010, at 4:40 , Paul Harris wrote:
>>>>>>>>>
>>>>>>>>>> On 7 July 2010 23:05, Michael Wild <themiwi at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 7. Jul, 2010, at 16:01 , Paul Harris wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> I have looked and can't find the answer, so I turn to the list.
>>>>>>>>>>>>
>>>>>>>>>>>> I have a CMakeLists.txt and a subdirectory called utils, which
>>>> also
>>>>>>> has
>>>>>>>>>>> its
>>>>>>>>>>>> own CMakeLists.txt
>>>>>>>>>>>>
>>>>>>>>>>>> In the parent CML.txt, I have something like:
>>>>>>>>>>>>
>>>>>>>>>>>> ENABLE_TESTING()
>>>>>>>>>>>> add_subdirectory(utils)
>>>>>>>>>>>>
>>>>>>>>>>>> In my utils CML.txt, I have
>>>>>>>>>>>>
>>>>>>>>>>>> ADD_EXECUTABLE(unit_1 units/unit_1.cpp)
>>>>>>>>>>>> ADD_TEST( unit_1 ${EXECUTABLE_OUTPUT_PATH}/unit_1 )
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Simplify this to
>>>>>>>>>>>
>>>>>>>>>>> ADD_TEST(unit_1 unit_1)
>>>>>>>>>>>
>>>>>>>>>>> CMake will figure out by itself that unit_1 is a target and
>> invoke
>>>> the
>>>>>>>>>>> executable correctly (your code would break for
>> multi-configuration
>>>>>>> IDE
>>>>>>>>>>> generators).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> This does not work for me.  If I do not have the
>>>> EXECUTABLE_OUTPUT_PATH
>>>>>>>>> in
>>>>>>>>>> add_test, I get a message like this when i run "make test"
>>>> (shortened
>>>>>>> for
>>>>>>>>>> brevity):
>>>>>>>>>>
>>>>>>>>>> 1/  1 Testing unit_string_numeric_utils     Could not find
>>>> executable
>>>>>>>>>> unit_string_numeric_utils
>>>>>>>>>> Looked in the following places:
>>>>>>>>>> unit_string_numeric_utils
>>>>>>>>>> unit_string_numeric_utils
>>>>>>>>>> Release/unit_string_numeric_utils
>>>>>>>>>> Release/unit_string_numeric_utils
>>>>>>>>>
>>>>>>>>> Mmmh, works fine for me:
>>>>>>>>>
>>>>>>>>> --------->8---------
>>>>>>>>> cmake_minimum_required(VERSION 2.8)
>>>>>>>>> project(tmp)
>>>>>>>>>
>>>>>>>>> enable_testing()
>>>>>>>>>
>>>>>>>>> add_executable(unit1 unit1.cpp)
>>>>>>>>> add_test(unit1 unit1)
>>>>>>>>> ---------<8---------
>>>>>>>>>
>>>>>>>>> Where unit1.cpp is just a simple hello-world program. Running it:
>>>>>>>>>
>>>>>>>>>
>>>>>>>> snip
>>>>>>>>
>>>>>>>> My project is a lot bigger than a hello-world program.  It has
>>>>>>>> subdirectories for a start, and I do things like
>>>>>>>> SET(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin CACHE INTERNAL
>>>>>>> "Single
>>>>>>>> output directory for building all executables.")
>>>>>>>>
>>>>>>>> I'm not sure at which point things stop working, do you want me to
>> try
>>>>>>> and
>>>>>>>> build a test-case?
>>>>>>>
>>>>>>> You are right, it seems that the documentation is misleading (or IMHO
>>>>>>> outright wrong). This, however, works for me and is safe:
>>>>>>>
>>>>>>> add_test(NAME unit1 COMMAND $<TARGET_FILE:unit1>)
>>>>>>>
>>>>>>> Note that NAME and COMMAND are required for this to work.
>>>>>>>
>>>>>>> BTW: EXECUTABLE_OUTPUT_PATH is deprecated, you should use
>>>>>>> CMAKE_RUNTIME_OUTPUT_DIRECTORY instead.
>>>>>>>
>>>>>>>
>>>>>> I changed EXE to that CMAKE RUNTIME thing, thanks.
>>>>>>
>>>>>> That NAME/COMMAND thing doesn't work for me at all.  Can't find the
>>>> binary
>>>>>> without the runtime output path explicitly added.
>>>>>>
>>>>>> This is what I tried:
>>>>>>
>>>>>> In parent CMakeLists.txt
>>>>>>
>>>>>> ENABLE_TESTING()
>>>>>>
>>>>>> set (CMAKE_TEST_COMMAND ctest -V)
>>>>>>
>>>>>> function (add_unit_test name)
>>>>>>   if(NOT TARGET ${name})
>>>>>>      add_custom_target (check COMMAND ${CMAKE_TEST_COMMAND})
>>>>>>   endif()
>>>>>>   add_executable(${name} ${ARGN})
>>>>>>   add_test(NAME ${name} COMMAND $<TARGET_FILE:${name}>)
>>>>>>   add_dependencies(check ${name})
>>>>>> endfunction()
>>>>>>
>>>>>>
>>>>>> in subdirectory CMakeLists.txt
>>>>>> add_unit_test(unit_string_numeric_utils
>>>> units/unit_string_numeric_utils.cpp
>>>>>> string_numeric_utils.cpp)
>>>>>
>>>>> Could you detect manually where the binary in question gets written
>>>>> to, and post ctest's output with the complaint about not finding it,
>>>>> and perhaps you could also post a minimal CMakeLists.txt file which
>>>>> demonstrates this issue along with the output of "make VERBOSE=1"?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Michael
>>>>
>>>> I suspect he's using a pre-2.8 version of CMake. I confirmed that
>> add_test
>>>> only resolves executable target names if used with NAME and COMMAND, and
>>>> this signature was introduced in 2.8.
>>>>
>>>>
>>> I'm using 2.8.1
>>> Michael Hertling - do you still want me to follow up as you asked?
>>
>> Yes, of course, if the problem still persists. Until now, I haven't
>> managed to let CTest miss an executable previously built by CMake,
>> i.e. ADD_TEST(NAME ... COMMAND ...) always worked well for me.
>>
>> Regards,
>>
>> Michael
>>
>>
> 
> Ok first thing: I had two cmakes on my system, one was 2.8.1, and an older
> 2.6.3 that snuck in through an incorrect script of mine.
> 
> Second thing: the TARGET_FILE thing works.  But the usual add_test(name
> command) does not.

This is because you set the CMAKE_RUNTIME_OUTPUT_DIRECTORY to a location
CMake does not take into account with ADD_TEST(<test> <exec>), but - as
MW already pointed out - ADD_TEST(NAME <test> COMMAND <target>) works
since this command resolves the executable's path for the target by
itself as $<TARGET_FILE:...> does, too.

Typically, ADD_TEST(<test> <exec>) would be used if the test driver is
an external tool, i.e. not part of your project, and located, e.g., by
FIND_PROGRAM(). Nevertheless, CMake looks for it at several familiar
places in case it's still built by the project. The recent signature
ADD_TEST(NAME <test> COMMAND <target>) is used if the test driver is
actually built by the project, and $<TARGET_FILE:...> et al. address
the difficulty to predict the location of binaries during the CMake
run when configuring for multi-configuration capable build systems.

In short: Take "add_test(NAME unit_1 COMMAND unit_1)" for Makefiles or
"add_test(NAME unit_1 COMMAND $<TARGET_FILE:unit_1>)" to be bulletproof.

Regards,

Michael