[CMake] Can't get CTEST_CUSTOM_ERROR_MATCH to work with CTest launchers

[Nils Gladitz]

I was able to reproduce the problem with Ninja on Ubuntu and CMake 
2.8.11-rc2.

I set up a test project for which I set CTEST_CUSTOM_ERROR_MATCH to 
"FooBar".

It has three custom commands with different outputs and exit codes:

     CustomCommand1: "this is a FooBar message" (exit success)
     CustomCommand2: "this is a fatal error" (exit failure)
     CustomCommand3: "this is a FooBar fatal error" (exit failure)

These are the results that I got (numbers in braces indicate which 
custom commands produce output visible on CDash):

     Ninja (Launchers enabled): 0 Build Errors
     Ninja (Launchers disabled): 4 Build Errors (1, 2, 3)

     Unix Makefiles (Launchers enabled): 2 Build Errors (2, 3)
     Unix Makefiles (Launchers disabled): 4 Build Errors (1, 2, 3)

For Unix Makefiles this seems to work better though not quite as I 
expected either.
(I expected CustomCommand1 to trigger an error even though the exit code 
is 0 due to it having FooBar in the output).

But if this is by design it would be nice to have at least the commands 
with failed exit status show up with Ninja like they do with Makefiles.

Hope this helps to diagnose the problem.

Nils

On 04/25/2013 07:04 PM, Jean-Christophe Fillion-Robin wrote:
> This is probably caused by commit 965358fcf [1]
>
> Can the problem be reproduced using Ninja on Unix platform ?
>
> [1] 
> https://github.com/Kitware/CMake/commit/965358fcf64cf1a3693bcdd66f723729e0614ef6
>
>
> On Thu, Apr 25, 2013 at 12:41 PM, Nils Gladitz <gladitz at sci-vis.de 
> <mailto:gladitz at sci-vis.de>> wrote:
>
>     Hey Jean-Christophe,
>
>     I currently use CMake 2.8.11-rc3 with the Ninja generator on windows.
>
>     Nils
>
>
>     On 25.04.2013 17 <tel:25.04.2013%2017>:30, Jean-Christophe
>     Fillion-Robin wrote:
>>     Hi Nils,
>>
>>     Since CTEST_USE_LAUNCHERS is ignored when used with generator
>>     different from "Make" or "Ninja"  [1], it seems strange that it
>>     impacts your windows build. Which generator are you using ?
>>
>>     Hth
>>     Jc
>>
>>     [1]
>>     https://github.com/Kitware/CMake/blob/master/Modules/CTestUseLaunchers.cmake#L38-40
>>
>>
>>     On Thu, Apr 25, 2013 at 4:44 AM, Nils Gladitz <gladitz at sci-vis.de
>>     <mailto:gladitz at sci-vis.de>> wrote:
>>
>>         One list entry in my CTEST_CUSTOM_ERROR_MATCH is "FAILED: "
>>         (in my CTestCustom.cmake.in <http://CTestCustom.cmake.in>
>>         which is used to generate CTestCustom.cmake in the build
>>         directory).
>>
>>         With CTEST_USE_LAUNCHERS enabled this does not seem to match
>>         errors from custom commands e.g.:
>>         "FAILED: cmd.exe /c " [...]
>>
>>         On my CDash installation I see 0 build errors.
>>
>>         When I disable CTEST_USE_LAUNCHERS the error shows up on the
>>         dashboard and the "FAILED: " line is highlighted indicating
>>         that a match has been made.
>>
>>         Is this by design (e.g. not implemented for launchers) or may
>>         I be missing something?
>>         Is there e.g. extra work required for
>>         CTEST_CUSTOM_ERROR_MATCH to reach the launcher?
>>
>>         Nils
>>
>>         -- 
>>         Nils Gladitz, B.Sc.
>>         DICOM, Konnektivität und Entwicklung
>>
>>         Scivis wissenschaftliche Bildverarbeitung GmbH
>>         Bertha-von-Suttner-Str. 5
>>         D-37085 Göttingen
>>         GERMANY
>>         Handelsregister Nr. / Trade Register No. B3100 Göttingen
>>         Geschäftsführer / Managing Directors Dr. Gernot Ebel, Dr. Uwe
>>         Engeland
>>
>>         Tel: 0049 (0)551 634181-28
>>         E-Mail: gladitz at scivis.de <mailto:gladitz at scivis.de>
>>         Web: www.scivis.de <http://www.scivis.de>
>>
>>         --
>>
>>         Powered by www.kitware.com <http://www.kitware.com>
>>
>>         Visit other Kitware open-source projects at
>>         http://www.kitware.com/opensource/opensource.html
>>
>>         Please keep messages on-topic and check the CMake FAQ at:
>>         http://www.cmake.org/Wiki/CMake_FAQ
>>
>>         Follow this link to subscribe/unsubscribe:
>>         http://www.cmake.org/mailman/listinfo/cmake
>>
>>
>>
>>
>>     -- 
>>     +1 919 869 8849 <tel:%2B1%20919%20869%208849>
>
>
>
>
> -- 
> +1 919 869 8849

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20130425/4584fb26/attachment-0001.htm>