[CMake] tests running out of order on cmake

[Michael Wild]

I agree that the information is not very discoverable if you've only
read the doc about add_test(), but it is there:

http://cmake.org/cmake/help/cmake-2-8-docs.html#command:set_tests_properties

http://cmake.org/cmake/help/cmake-2-8-docs.html#section_PropertiesonTests

Michael

On 06/28/2011 04:52 PM, j s wrote:
> On Tue, Jun 28, 2011 at 9:46 AM, j s <j.s4403 at gmail.com> wrote:
>> On Tue, Jun 28, 2011 at 9:33 AM, Michael Wild <themiwi at gmail.com> wrote:
>>> If the tests depend on each other, you *must* tell CTest so, by setting
>>> the DEPENDS property using the set_tests_properties() function.
>>>
>>> Normally, running tests in arbitrary order is considered to be a
>>> feature, helping you to not to introduce accidental dependencies.
>>
>> I respectfully disagree.  It is a bug, which had been marked as major
>> severity, as noted by the link in my previous email.  I have 85
>> regressions, and the first ones run the fastest, so I immediately know
>> if there is a problem.  Now that I am aware of set_test_properties,
>> I'll set them for the tests that need them.  When not running tests in
>> parallel, which is always, they need to run in the exact order
>> specified.  It is only a feature, if there is a default option like
>> schedule_sequential, which allows the program to work as I had come to
>> expect over the past 6 years.
> 
> It is also a bug since the man page for cmake 2.8.4 does not mention
> setting dependencies with set_test_properties.
> 
>>
>> Juan
>>
>>>
>>> Michael
>>>
>>> On 06/28/2011 04:16 PM, j s wrote:
>>>> Can someone please let me know how to get the Mac OS X version of
>>>> cmake 2.8.4 to run the tests in the order in which they are added?
>>>> For some bizarre reason they are running out of order:
>>>>
>>>> Sample output (without actual test name).
>>>>  1/85 Test #64: XXXXXXXXXXXXXXXX .............***Failed    0.01 sec
>>>>       Start 63: XXXXXXXXXXXXXXXX
>>>>  2/85 Test #63: XXXXXXXXXXXXXXXX..............   Passed    0.00 sec
>>>>       Start 62: XXXXXXXXXXXXXXXX
>>>>  3/85 Test #62: XXXXXXXXXXXXXXXX..............   Passed    0.00 sec
>>>>       Start 61: XXXXXXXXXXXXXXXX
>>>>
>>>> This is preventing from verifying my recent changes on the mac.  The
>>>> Ubuntu linux version does not have this issue.
>>>>
>>>> Juan
>>>>
>>>>
>>>> On Sat, Jun 25, 2011 at 2:51 PM, j s <j.s4403 at gmail.com> wrote:
>>>>> Unfortunately the problem came back.  I there a command line option to
>>>>> fix this out of order execution feature.
>>>>>
>>>>> Juan
>>>>>
>>>>> On Sat, Jun 25, 2011 at 1:30 PM, j s <j.s4403 at gmail.com> wrote:
>>>>>> I was able to update the cmake dmg from the website, and now my tests
>>>>>> are running in the proper order.
>>>>>>
>>>>>> On Sat, Jun 25, 2011 at 1:23 PM, j s <j.s4403 at gmail.com> wrote:
>>>>>>> How do I tell the macports version of cmake 2.8.1  on the mac to run
>>>>>>> the tests in exactly the order they were originally specified?  I'm in
>>>>>>> the process of updating macports right now to see if it goes back to
>>>>>>> the sequential default in later versions?  man ctest reveals
>>>>>>> --schedule-random, but no sequential option.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Juan
>>>>>>>
>>>>>>
>>>>>
>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://www.cmake.org/mailman/listinfo/cmake
>>>
>>