[cmake-developers] [CMake] How to set _default_ timeout for the ctest command? (fwd)

[David Cole]

Sounds to me like lapack should conditionally set CTEST_TEST_TIMEOUT only if it's not DEFINED already. With such code in the project, callers could define it on the command line with -D, or directly in a ctest -S script, and the passed in vale would "win" since in this case, the project would NOT set its default value.

Pretty standard to do "set only if not set" for variables that callers might potentially want to override in a ctest script or in a CMakeLists file.

This works with existing recent versions of cmake and ctest just fine... I think ctest has supported -D cmd line params since 3.0 or so.


David

> On Dec 17, 2015, at 6:38 PM, Alan W. Irwin <irwin at beluga.phys.uvic.ca> wrote:
> 
>> On 2015-12-17 13:54-0500 David Cole wrote:
>> 
>> I agree with Brad, the --timeout command line parameter should only
>> set/override the variable CTEST_TEST_TIMEOUT. The behavior w.r.t. test
>> TIMEOUT properties should be left as is for this change.
>> 
>> A **new** --timeout-scale with well defined / documented interactions
>> with the global variable and the test properties would be the best
>> approach for introducing changes in behavior. Or a **new**
>> --ignore-timeout-test-property to simply use the global value
>> everywhere for some use cases.
> 
> This seems like a reasonable approach to me to get these timeout issues
> addressed.
> 
> However, to be clear, here are the consequences for a project like
> lapack where some extrordinary but legitimate timing variations can occur.
> 
> 1. The current advice to such projects would be to rigourously avoid setting either
> CTEST_TEST_TIMEOUT or individual test timeout properties since the user currently
> has no control over those (which illustrates why addressing the above
> issues is important).
> 
> 2. Once the --timeout fix that allows it to override
> CTEST_TEST_TIMEOUT set by the project has become part of a release,
> AND when that version of cmake is the minimum allowed by the project,
> then it will be safe for the project to start using CTEST_TEST_TIMEOUT
> since that value can now be overridden by the user with the --timeout
> option. But the project should still rigourously avoid using
> individual timeout properties until the conditions listed in 3. are
> true.
> 
> 3. Once the implementation of the proposed new --timeout-scale option
> has been implemented and become part of a release, AND when that
> version of cmake is the minimum allowed by the project, then it will
> be safe for such projects to set both CTEST_TEST_TIMEOUT and
> individual timeout properties since the user now has full control
> over all such project timeouts.
> 
> Alan
> __________________________
> Alan W. Irwin
> 
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
> 
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
> 
> Linux-powered Science
> __________________________