[cmake-developers] [Patch] Add option "--no-return-code"

A. Klitzing aklitzing at gmail.com
Tue Aug 18 04:30:13 EDT 2015


Yeah, you are right. If ctest itself fails it should return non-zero.

We use qtestlib [1] from Qt5 and add every testsuite with ADD_TEST and dump
a logfile that will be parsed by xunit plugin [2] of jenkins.

ADD_TEST(${TESTNAME} ${TESTNAME} -platform offscreen -v2 -o
${CMAKE_CURRENT_BINARY_DIR}/results.${TESTNAME}.log.xml,xml)

The problem is that the test binaries of qtestlib will return non-zero if
some testcase is broken. So ctest will also return non-zero because a
testsuite "failed".
Of course, this is the correct behaviour if something goes wrong. But if
ctest returns non-zero, jenkins will abort the whole build instead of let
xunit plugin mark the build as "unstable" because testcase XY failed.

Best regards
   André


[1] http://doc.qt.io/qt-5/qtest-overview.html
[2] https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin



2015-08-14 16:01 GMT+02:00 Brad King <brad.king at kitware.com>:

> On 08/13/2015 04:09 PM, A. Klitzing wrote:
> > We use ctest on Jenkins in a shell build step. Jenkins will FAIL
> > the build if the last command (ctest) will return a non-zero
> > return code even the configured UNSTABLE/FAIL threshold is not
> > reached.
> >
> > This won't be necessary if ctest could ignore the return codes.
>
> If something goes wrong with the administration of running the
> dashboard client, like "git fetch" fails, then CTest should report
> failure via a non-zero return code.  Some cleanup of this was done
> recently to tolerate failed tests and such without making the process
> exit code non-zero.  If you've found remaining cases where normal
> testing failures produce a non-zero exit from ctest then we should
> fix those first.
>
> Can you explain your case in more detail, please?
>
> Thanks,
> -Brad
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20150818/96c2bcaf/attachment.html>


More information about the cmake-developers mailing list