[cmake-developers] [CMake 0015599]: CTest with launchers drops warning generated by early build command failure

[Mantis Bug Tracker]

The following issue has been SUBMITTED. 
====================================================================== 
http://www.cmake.org/Bug/view.php?id=15599 
====================================================================== 
Reported By:                trsystran
Assigned To:                
====================================================================== 
Project:                    CMake
Issue ID:                   15599
Category:                   CTest
Reproducibility:            always
Severity:                   minor
Priority:                   normal
Status:                     new
====================================================================== 
Date Submitted:             2015-06-01 05:35 EDT
Last Modified:              2015-06-01 05:35 EDT
====================================================================== 
Summary:                    CTest with launchers drops warning generated by
early build command failure
Description: 
When the build command fails (returns a non-zero return code), CTest creates a
~fake warning in cmCTestBuildHandler::RunMakeCommand:
"this->ErrorsAndWarnings.push_back(errorwarning);".

(Sub-question: Why is it a Warning and not an Error? all non-zero return code
should be errors)

Then, normally the "ErrorsAndWarnings" object is parsed and converted to
"Build.xml" in cmCTestBuildHandler::GenerateXMLLogScraped.
However, when CTest is run with Launchers enabled, then GenerateXMLLogScraped is
never called: cmCTestBuildHandler::GenerateXMLLaunched is called instead, using
a different source of information for errors and warnings: the xml files
generated by ctest in launcher mode.

This means that the build return code value is ignored.


Why is it an issue when we have launchers?
Because launchers only catch build errors as they wrap the compiler command.
If the generator fails early (ie before executing any build command), then the
ctest launchers won't catch anything here.

It happens with the Ninja generator, when we import an external lib, but forcing
an invalid path to the lib.

Steps to Reproduce: 
$ cmake . -G Ninja -DCTEST_USE_LAUNCHERS=ON
$ ninja
ninja: error: '/invalid/path/to/libmissingLib.so', needed by 'executableFailed',
missing and no known rule to make it

No call to ctest in launcher mode confirmed with "strace -f ninja |& grep
ctest".


When launched via a real ctest flow:
Build project
MakeCommand:ninja -k0 -l34 all
Run command: "ninja" "-k0" "-l34" "all"
   Each symbol represents 1024 bytes of output.
    .ninja: error: '/invalid/path/to/libmissingLib.so', needed by
'executableFailed', missing and no known rule to make it
 Size of output: 0K
Command exited with the value: 1
MakeCommand:ninja -k0 -l34 all
Error(s) when building project
   0 Compiler errors
   1 Compiler warnings

But the Build.xml has no error nor warning: no issue is pushed to cdash.

Additional Information: 
There are probably many other cases of early generator failure (for example if a
source file goes missing between cmake configuration and build), at least with
Ninja.
====================================================================== 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2015-06-01 05:35 trsystran      New Issue                                    
2015-06-01 05:35 trsystran      File Added: CMakeLists.txt                    
======================================================================