[cmake-developers] [CMake] How to set _default_ timeout for the ctest command? (fwd)

[Alan W. Irwin]

On 2015-12-17 13:54-0500 David Cole wrote:

> I agree with Brad, the --timeout command line parameter should only
> set/override the variable CTEST_TEST_TIMEOUT. The behavior w.r.t. test
> TIMEOUT properties should be left as is for this change.
>
> A **new** --timeout-scale with well defined / documented interactions
> with the global variable and the test properties would be the best
> approach for introducing changes in behavior. Or a **new**
> --ignore-timeout-test-property to simply use the global value
> everywhere for some use cases.

This seems like a reasonable approach to me to get these timeout issues
addressed.

However, to be clear, here are the consequences for a project like
lapack where some extrordinary but legitimate timing variations can occur.

1. The current advice to such projects would be to rigourously avoid setting either
CTEST_TEST_TIMEOUT or individual test timeout properties since the user currently
has no control over those (which illustrates why addressing the above
issues is important).

2. Once the --timeout fix that allows it to override
CTEST_TEST_TIMEOUT set by the project has become part of a release,
AND when that version of cmake is the minimum allowed by the project,
then it will be safe for the project to start using CTEST_TEST_TIMEOUT
since that value can now be overridden by the user with the --timeout
option. But the project should still rigourously avoid using
individual timeout properties until the conditions listed in 3. are
true.

3. Once the implementation of the proposed new --timeout-scale option
has been implemented and become part of a release, AND when that
version of cmake is the minimum allowed by the project, then it will
be safe for such projects to set both CTEST_TEST_TIMEOUT and
individual timeout properties since the user now has full control
over all such project timeouts.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________