[CMake] CTest/CDash and miscellaneous post-build steps (was Re: continuous integration with CMake)

[Clinton Stimpson]

I do this to get test results:

# find dir containing testing files
FILE(READ "${CTEST_BINARY_DIRECTORY}/Testing/TAG" TAG_FILE)
STRING(REGEX MATCH "[^\n]*" TEST_FILE_DIR ${TAG_FILE})
SET(TEST_FILE_DIR "${CTEST_BINARY_DIRECTORY}/Testing/${TEST_FILE_DIR}")

SET(test_passed 0)
CTEST_TEST(BUILD "${CTEST_BINARY_DIRECTORY}" ${CTEST_TEST_OPTIONS} 
RETURN_VALUE test_result)

# test_results contains whether ctest was able to run tests, 
# not whether any tests failed.
# read test results file for test failures

  IF(test_result MATCHES 0)
    FILE(READ "${TEST_FILE_DIR}/Test.xml" TEST_RESULTS)
    STRING(REGEX MATCH "Status=\"[^p]" TestErrors "${TEST_RESULTS}")
    IF(TestErrors MATCHES "")
      SET(test_passed 1)
    ENDIF(TestErrors MATCHES "")
  ENDIF(test_result MATCHES 0)


The to do things based on test results, I do:
IF(test_passed)
    EXECUTE_PROCESS(COMMAND  some stuff.... )
ENDIF(test_passed)

The stuff I do includes packaging, copying new packages to other machines on 
the network, etc...  I've found ssh+keys does a good job.

I've submitted a request to make it easier to do operations based on results 
from the update/configure/build/test phases.
http://public.kitware.com:80/Bug/view.php?id=8277

Clint


On Wednesday 17 February 2010 02:58:18 pm Tyler Roscoe wrote:
> Next question:
>
> - CDash and miscellaneous post-build steps
>
> Now that I've got Continuous Integration up and running with CDash, I'd
> like to publish my Continuous builds to a central build archive so
> developers and testers can use them.
>
> I can write post-build rules to copy my _results directory to the build
> archive (or mess with CMAKE_INSTALL_PREFIX and the INSTALL target), but
> then the build will be copied as soon as compiling/linking are finished.
> I would prefer to run the tests first and only copy the build if it
> passes all the tests.
>
> I thought of doing the copy from my continuous integration script
> itself. I could copy after a successful build by making use of the
> NUMBER_ERRORS and NUMBER_WARNINGS parameters to ctest_build(). I was
> hoping I could do the same after ctest_test(), but that command doesn't
> seem to have anything like NUMBER_ERRORS that would let me evaluate if
> the tests completed successfully before performing the copy.
>
> How do other people handle this?
>
> I'd like to do more than just publish the builds (run the setup.exe and
> make sure it works, compare what is installed by setup.exe against a
> known-good install manifest, kick off component tests on another machine
> with this newly-generated build, etc.) but figuring out a method for
> publishing builds would be a great start.
>
> Thanks,
> tyler
>
> On Fri, Dec 04, 2009 at 01:07:23PM -0500, Bill Hoffman wrote:
> > Tyler Roscoe wrote:
> > The build scripts for CMake can be found on CDash itself, for example:
> >
> > http://www.cdash.org/CDash/viewNotes.php?buildid=485762
> >
> > For each dashboard on the CMake dashboard if you click on the notes for
> > the build you can see the script that was used to drive the build.
> >
> > Here is the .bat file that is run for a scheduled task on that machine:
> >
> >
> > $ cat dash2win64.bat
> > setlocal
> > rem Update clapack driver script from clapck
> > pushd c:\Dashboards\clapack
> > svn up clapack_build.cmake
> > rem Run the clapack driver script
> > "c:\Program Files (x86)\CMake 2.8\bin\ctest.exe" -C Release -S
> > clapack_build.cmake -VV -O clapack.log
> > popd
> > endlocal
> >
> > call C:\cygwin\bin\bash.exe --login
> > c:/Dashboards/DashboardScripts/dash2win64cygwin.sh
> >
> > "c:\Program Files (x86)\CMake 2.8\bin\ctest.exe" -C Release -S
> > dash2Win64_cmake_vs10.cmake -O cmake_vs10.log
> > "c:\Program Files (x86)\CMake 2.8\bin\ctest.exe" -C Release -S
> > dash2Win64_cmake_vs10_x64.cmake -O cmake_vs10_x64.log
> >
> > setlocal
> > call "C:\Program Files (x86)\Microsoft Visual Studio
> > 10.0\VC\bin\vcvars32.bat"
> > "c:\Program Files (x86)\CMake 2.8\bin\ctest.exe" -C Release -S
> > dash2Win64_cmake_nmake10.cmake -O cmake_nmake10.log
> > endlocal
> >
> > setlocal
> > call "C:\Program Files (x86)\Microsoft Visual Studio
> > 10.0\VC\bin\amd64\vcvars64.bat"
> > "c:\Program Files (x86)\CMake 2.8\bin\ctest.exe" -C Release -S
> > dash2Win64_cmake_nmake10_x64.cmake -O cmake_nmake10_x64.log
> > endlocal
> >
> > "c:\Program Files (x86)\CMake 2.8\bin\ctest.exe" -C Release -S
> > dash2win64_libarchive_vs9.cmake -VV > libarchive_vs9.log 2>&1
> > "c:\Program Files (x86)\CMake 2.9\bin\ctest.exe" -C Release -S
> > dash2win64_cmake_icl11_32.cmake -VV > cmake_icl11_32.log 2>&1
> > "c:\Program Files (x86)\CMake 2.9\bin\ctest.exe" -C Release -S
> > dash2win64_cmake_icl11_64.cmake -VV > cmake_icl11_64.log 2>&1
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake