Don't worry... it's not submitting results to Kitware, it's using a default value for CTEST_TRIGGER_SITE, which is old and leftover from pre-CDash days... It used to be the case that after all results were uploaded, a "trigger" script would run to tell the server to process the previously uploaded files. CDash processes files as they are submitted so triggers are not necessary with CDash. I actually fixed this very code in ctest earlier today to avoid trigger calls when using CDash. (See changes listed here: <a href="http://www.cdash.org/CDash/viewUpdate.php?buildid=263494">http://www.cdash.org/CDash/viewUpdate.php?buildid=263494</a>)<br>
<br><div>To answer your questions:</div><div>The CMake book has a Tutorial chapter in it that takes you through the steps involved in adding CTest / CDash support to your CMake based project. If you have the book, read through the Tutorial chapter... it's pretty quick. If you don't have the book, buy it. :-) (You can also check out the code that goes with the chapter in the CMake/Tests/Tutorial/Step1 through Step7 directories in a CMake source tree...)</div>
<div><br></div><div>Basically, you need to call ENABLE_TESTING() and INCLUDE(CTest) in your CMakeLists.txt file.</div><div><br></div><div>And yes, "make test" returns errors when there are test failures...</div>
<div><br></div><div><div>And you should set CTEST_UPDATE_COMMAND to the full path to the svn executable to use svn instead of cvs...</div><div><br></div><div><br></div><div>HTH,</div><div>David</div><div><br></div><div><br>
</div></div><div><div class="gmail_quote">On Tue, Feb 3, 2009 at 3:13 PM, Bill O'Hara <span dir="ltr"><<a href="mailto:billtohara@gmail.com">billtohara@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hello,<br><br>I wonder if anyone can point me in the direction of a hello world project that successfully demonstrates the basic features of cmake+ctest+cdash? I'm using cmake/ctest version 2.6-patch 2 and cdash 1.2.1. Or somewhere with a working tutorial I can follow along? (I'm a bit confused by where to find the up to date documentation I should work from).<br>
<br>I'm tasked with putting together a new build system for a commercial project but when evaluating cmake I can't seem to get the basics working even on trivial hello world projects. The comedy of errors I've managed so far is included below - I'd be extremely grateful for any hints. I'm particularly worried by the testing submitting results to <a href="http://kitware.com" target="_blank">kitware.com</a> even though I specifically said we're using cdash and pointed it at my local cdash server (where some results are submitted despite other errors).<br>
<br>Thanks in advance for any hints! I have marked QUESTION below at various points if anyone can spot my problems :(<br><br>Bill<br><br><br>#<br># Start from a clean setup.<br>#<br>bill-box:/tmp> ls svn<br>README.txt conf dav db format hooks locks<br>
bill-box:/tmp> ls build/<br>bill-box:/tmp> ls binary/<br>bill-box:/tmp> ls source/<br><br><br>#<br># Checkout has a minimal test project checked out from /tmp/svn:<br># - A shared library called impl build from include/header.h and impl/impl.c<br>
# - A test called foo that invokes the function from impl and always succeeds.<br># - A test called bar that invokes the function from impl and always fails.<br>#<br>bill-box:/tmp> ls checkout/<br>CMakeLists.txt CTestConfig.cmake impl include tests<br>
<br>#<br># CMakeLists.txt<br>#<br>bill-box:/tmp/checkout> cat CMakeLists.txt<br>
PROJECT (HELLOWORLD)<br>
CMAKE_MINIMUM_REQUIRED(VERSION 2.6)<br>
INCLUDE_DIRECTORIES (${CMAKE_SOURCE_DIR}/include)<br>
ADD_SUBDIRECTORY (${CMAKE_SOURCE_DIR}/impl)<br>
INCLUDE (CTest)<br>
ADD_SUBDIRECTORY (${CMAKE_SOURCE_DIR}/tests)<br>
<br><br>#<br># CTestConfig.cmake<br>#<br>
bill-box:/tmp/checkout> cat CTestConfig.cmake<br>
SET (CTEST_DROP_SITE_CDASH TRUE)<br>
SET (CTEST_PROJECT_NAME "helloworld")<br>
SET (CTEST_NIGHTLY_START_TIME "00:00:00 EST")<br>
SET (UPDATE_TYPE "true")<br>
SET (CTEST_DROP_METHOD "http")<br>
SET (CTEST_DROP_SITE "bill-box")<br>
SET (CTEST_DROP_LOCATION "/submit.php?project=helloworld")<br>
<br>
<br>#<br># And an experimental Nightly ctest script to try to upload<br># test results to CDash after checking out and building<br># the project source code. This script is not in /tmp/checkout.<br>#<br>bill-box:/tmp> cat experimental.cmake<br>
SET (CTEST_DROP_SITE_CDASH TRUE)<br>
SET (CTEST_PROJECT_NAME "helloworld")<br>
SET (CTEST_NIGHTLY_START_TIME "00:00:00 EST")<br>
SET (UPDATE_TYPE "true")<br>
SET (CTEST_DROP_METHOD "http")<br>
SET (CTEST_DROP_SITE "bill-box")<br>
SET (CTEST_DROP_LOCATION "/submit.php?project=helloworld")<br>
SET (CTEST_SOURCE_DIRECTORY "/tmp/source")<br>
SET (CTEST_BINARY_DIRECTORY "/tmp/binary")<br>
SET (CTEST_START_WITH_EMPTY_BINARY_DIRECTORY TRUE)<br>
SET (CTEST_SVN_CHECKOUT "svn co file:///tmp/svn \"${CTEST_SOURCE_DIRECTORY}\"")<br>
SET (CTEST_CMAKE_GENERATOR "Unix Makefiles")<br>
SET (CTEST_CUSTOM_PRE_TEST "${CTEST_SVN_CHECKOUT}")<br>
CTEST_START(Nightly)<br>
CTEST_UPDATE(SOURCE "${CTEST_SOURCE_DIRECTORY}")<br>
CTEST_CONFIGURE(BUILD "${CTEST_BINARY_DIRECTORY}")<br>
CTEST_BUILD(BUILD "${CTEST_BINARY_DIRECTORY}")<br>
CTEST_TEST(BUILD "${CTEST_BINARY_DIRECTORY}")<br>
CTEST_SUBMIT()<br>
<br><br>#<br># First, try to configure using cmake.<br>#<br>bill-box:/tmp> cd build<br>bill-box:/tmp/build> cmake /tmp/checkout<br>-- The C compiler identification is GNU<br>-- The CXX compiler identification is GNU<br>
-- Check for working C compiler: /usr/bin/gcc<br>-- Check for working C compiler: /usr/bin/gcc -- works<br>-- Detecting C compiler ABI info<br>-- Detecting C compiler ABI info - done<br>-- Check for working CXX compiler: /usr/bin/c++<br>
-- Check for working CXX compiler: /usr/bin/c++ -- works<br>-- Detecting CXX compiler ABI info<br>-- Detecting CXX compiler ABI info - done<br>-- Configuring done<br>-- Generating done<br>-- Build files have been written to: /tmp/build<br>
<br><br>#<br># Now, build and test.<br>#<br>bill-box:/tmp/build> make all test<br>Scanning dependencies of target impl<br>[ 33%] Building C object impl/CMakeFiles/impl.dir/impl.c.o<br>Linking C shared library libimpl.so<br>
[ 33%] Built target impl<br>Scanning dependencies of target bar<br>[ 66%] Building C object tests/CMakeFiles/bar.dir/bar.c.o<br>Linking C executable bar<br>[ 66%] Built target bar<br>Scanning dependencies of target foo<br>
[100%] Building C object tests/CMakeFiles/foo.dir/foo.c.o<br>Linking C executable foo<br>[100%] Built target foo<br>Running tests...<br>Start processing tests<br>Test project /tmp/build<br> 1/ 2 Testing foo Passed<br>
2/ 2 Testing bar ***Failed<br><br>50% tests passed, 1 tests failed out of 2<br><br>The following tests FAILED:<br> 2 - bar (Failed)<br>Errors while running CTest<br>make: *** [test] Error 8<br>
<br><br># QUESTION<br>#<br># Is this Error 8 from make the expected result after some tests have failed?<br>#<br><br><br>#<br># Now try the Nightly test target.<br>#<br>bill-box:/tmp/build> make Nightly<br>Scanning dependencies of target Nightly<br>
Site: bill-box<br> Build name: Linux-c++<br>Determine Nightly Start Time<br> Specified time: 00:00:00 EST<br>Create new tag: 20090203-0500 - Nightly<br>Start processing tests<br><br>#<br># QUESTION<br># Why does it think we are using CVS here?<br>
# <br><br>Updating the repository<br> Updating the repository: /tmp/checkout<br> Use CVS repository type<br>Determine Nightly Start Time<br> Specified time: 00:00:00 EST<br> Gathering version information (each . represents one updated file):<br>
Project is up-to-date<br> Update with command: "/usr/bin/cvs" -z3 update -d -A -P -D "2009-02-03 05:00:00 UTC" failed<br>Configure project<br> Each . represents 1024 bytes of output<br> . Size of output: 0K<br>
Build project<br> Each symbol represents 1024 bytes of output.<br> '!' represents an error and '*' a warning.<br> . Size of output: 0K<br> 0 Compiler errors<br> 0 Compiler warnings<br>Test project /tmp/build<br>
1/ 2 Testing foo Passed<br> 2/ 2 Testing bar ***Failed<br><br>50% tests passed, 1 tests failed out of 2<br><br>The following tests FAILED:<br> 2 - bar (Failed)<br>
<br>#<br># QUESTION<br># How do I tell it to not do Coverage?<br>#<br><br>Performing coverage<br> Cannot find any coverage files. Ignoring Coverage request.<br>Submit files (using http)<br> Using HTTP submit method<br>
Drop site: <a href="http://bill-box/submit.php?project=helloworld" target="_blank">http://bill-box/submit.php?project=helloworld</a><br>
Uploaded: /tmp/build/Testing/20090203-0500/Build.xml<br> Uploaded: /tmp/build/Testing/20090203-0500/Configure.xml<br> Uploaded: /tmp/build/Testing/20090203-0500/Test.xml<br> Uploaded: /tmp/build/Testing/20090203-0500/Update.xml<br>
Submission successful<br>Errors while running CTest<br>make[3]: *** [CMakeFiles/Nightly] Error 8<br>make[2]: *** [CMakeFiles/Nightly.dir/all] Error 2<br>make[1]: *** [CMakeFiles/Nightly.dir/rule] Error 2<br>make: *** [Nightly] Error 2<br>
<br>#<br># QUESTION<br># Why are we getting 3 different Errors from make?<br>#<br><br><br>#<br>#<br># Let us try that again using ctest with verbosity turned on.<br>#<br>#<br><br>bill-box:/tmp/build> ctest -VV<br><br>
#<br>
# QUESTION<br># Where is this DartConfiguration coming from? We're trying<br># to use CDash.<br>#<br><br>UpdateCTestConfiguration from :/tmp/build/DartConfiguration.tcl<br>Parse Config file:/tmp/build/DartConfiguration.tcl<br>
Start processing tests<br>UpdateCTestConfiguration from :/tmp/build/DartConfiguration.tcl<br>Parse Config file:/tmp/build/DartConfiguration.tcl<br>Test project /tmp/build<br>Constructing a list of tests<br>Done constructing a list of tests<br>
Changing directory into /tmp/build/tests<br> 1/ 2 Testing foo<br>Test command: /tmp/build/tests/foo foo.c<br>Test timeout computed to be: 1500<br>42<br>-- Process completed<br> Passed<br> 2/ 2 Testing bar<br>Test command: /tmp/build/tests/bar bar.c<br>
Test timeout computed to be: 1500<br>42<br>-- Process completed<br>***Failed<br><br>50% tests passed, 1 tests failed out of 2<br><br>The following tests FAILED:<br> 2 - bar (Failed)<br>Errors while running CTest<br>
<br>#<br>#<br># Try our experimental Nightly test script to check out and test the project.<br>#<br>#<br><br>bill-box:/tmp> ctest -S experimental.cmake<br><br>#<br># Drat; its complaining about the source control system and trying<br>
# to build and update before running my svn checkout successfully.<br>#<br># It also fails to find my tests.<br>#<br><br>Cannot find CVSCommand, SVNCommand, or UpdateCommand key in the DartConfiguration.tcl<br>Error(s) when updating the project<br>
Error(s) when building project<br>A /tmp/source/impl<br>A /tmp/source/impl/impl.c<br>A /tmp/source/impl/CMakeLists.txt<br>A /tmp/source/tests<br>A /tmp/source/tests/foo.c<br>A /tmp/source/tests/bar.c<br>A /tmp/source/tests/CMakeLists.txt<br>
A /tmp/source/include<br>A /tmp/source/include/header.h<br>A /tmp/source/CTestConfig.cmake<br>A /tmp/source/experimental.cmake<br>A /tmp/source/CMakeLists.txt<br>Checked out revision 9.<br>No tests were found!!!<br>
<br>
#<br># QUESTION<br># Why were my tests not found?<br><br>#<br># <br># Try that again with verbosity; the previous attempt seems to have<br># helped by eventually checking out source code.<br>#<br>#<br><br>bill-box:/tmp> ctest -VV -S experimental.cmake<br>
* Extra verbosity turned on<br>Reading Script: /tmp/experimental.cmake<br>SetCTestConfiguration:SourceDirectory:/tmp/source<br>SetCTestConfiguration:BuildDirectory:/tmp/binary<br>Run dashboard with model Nightly<br> Source directory: /tmp/source<br>
Build directory: /tmp/binary<br> Reading ctest configuration file: /tmp/source/CTestConfig.cmake<br>SetCTestConfigurationFromCMakeVariable:NightlyStartTime:CTEST_NIGHTLY_START_TIMESetCTestConfiguration:NightlyStartTime:00:00:00 EST<br>
Site:<br> Build name:<br>Determine Nightly Start Time<br> Specified time: 00:00:00 EST<br> Use Nightly tag: 20090203-0500<br>SetCTestConfiguration:SourceDirectory:/tmp/source<br>SetCTestConfiguration:BuildDirectory:/tmp/binary<br>
SetCTestConfiguration:SourceDirectory:/tmp/source<br>Updating the repository<br> Updating the repository: /tmp/source<br>Cannot find CVSCommand, SVNCommand, or UpdateCommand key in the DartConfiguration.tcl<br>SetCTestConfiguration:BuildDirectory:/tmp/binary<br>
SetCTestConfiguration:SourceDirectory:/tmp/source<br>SetCTestConfiguration:ConfigureCommand:"/opt/cmake-2.6.2/bin/cmake" "-GUnix Makefiles" "/tmp/source"<br>SetCTestConfiguration:BuildDirectory:/tmp/binary<br>
SetCTestConfiguration:SourceDirectory:/tmp/source<br>Configure project<br>Configure with command: "/opt/cmake-2.6.2/bin/cmake" "-GUnix Makefiles" "/tmp/source"<br>Run command: "/opt/cmake-2.6.2/bin/cmake" "-GUnix Makefiles" "/tmp/source"<br>
Each . represents 1024 bytes of output<br> .-- Configuring done<br>-- Generating done<br>-- Build files have been written to: /tmp/binary<br> Size of output: 0K<br>Command exited with the value: 0<br>SetMakeCommand:/usr/bin/gmake -i<br>
SetCTestConfiguration:MakeCommand:/usr/bin/gmake -i<br>SetCTestConfiguration:BuildDirectory:/tmp/binary<br>SetCTestConfiguration:SourceDirectory:/tmp/source<br>Build project<br>MakeCommand:/usr/bin/gmake -i<br>Run command: "/usr/bin/gmake" "-i"<br>
Each symbol represents 1024 bytes of output.<br> '!' represents an error and '*' a warning.<br> .[ 33%] Built target impl<br>[ 66%] Built target bar<br>[100%] Built target foo<br> Size of output: 0K<br>
Command exited with the value: 0<br> 0 Compiler errors<br> 0 Compiler warnings<br>SetCTestConfiguration:BuildDirectory:/tmp/binary<br>SetCTestConfiguration:SourceDirectory:/tmp/source<br>Test project /tmp/binary<br>Run command: svn co file:///tmp/svn "/tmp/source"<br>
Checked out revision 9.<br>Constructing a list of tests<br>Changing directory into /tmp/binary/tests<br> 1/ 2 Testing foo<br>Test command: /tmp/binary/tests/foo foo.c<br>Test timeout computed to be: 600<br>42<br>-- Process completed<br>
Passed<br> 2/ 2 Testing bar<br>Test command: /tmp/binary/tests/bar bar.c<br>Test timeout computed to be: 600<br>42<br>-- Process completed<br>***Failed<br><br>50% tests passed, 1 tests failed out of 2<br><br>The following tests FAILED:<br>
2 - bar (Failed)<br>* Use default trigger site: <a href="http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi" target="_blank">http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi</a><br>
SetCTestConfiguration:DropMethod:http<br>
SetCTestConfiguration:DropSite:bill-box<br>SetCTestConfiguration:DropLocation:/submit.php?project=helloworld<br>SetCTestConfiguration:TriggerSite:<a href="http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi" target="_blank">http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi</a><br>
SetCTestConfiguration:BuildDirectory:/tmp/binary<br>SetCTestConfiguration:SourceDirectory:/tmp/source<br>Submit files (using http)<br> Using HTTP submit method<br> Drop site: <a href="http://bill-box/submit.php?project=helloworld" target="_blank">http://bill-box/submit.php?project=helloworld</a><br>
Upload file: /tmp/binary/Testing/20090203-0500/Build.xml to <a href="http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Build.xml" target="_blank">http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Build.xml</a> Size: 909<br>
Uploaded: /tmp/binary/Testing/20090203-0500/Build.xml<br> Upload file: /tmp/binary/Testing/20090203-0500/Configure.xml to <a href="http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Configure.xml" target="_blank">http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Configure.xml</a> Size: 1066<br>
Uploaded: /tmp/binary/Testing/20090203-0500/Configure.xml<br> Upload file: /tmp/binary/Testing/20090203-0500/Test.xml to <a href="http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Test.xml" target="_blank">http://bill-box/submit.php?project=helloworld&FileName=______20090203-0500-Nightly___XML___Test.xml</a> Size: 2272<br>
Uploaded: /tmp/binary/Testing/20090203-0500/Test.xml<br>#<br># QUESTION<br># What the...?<br>#<br># Why is it submitting results to <a href="http://kitware.com" target="_blank">kitware.com</a>?!<br>#<br>#<br>#<br> Using HTTP trigger method<br>
Trigger site: <a href="http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi" target="_blank">http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi</a><br> Trigger url: <a href="http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Build.xml" target="_blank">http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Build.xml</a><br>
<br> Trigger url: <a href="http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Configure.xml" target="_blank">http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Configure.xml</a><br>
<br> Trigger url: <a href="http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Test.xml" target="_blank">http://public.kitware.com/cgi-bin/Submit-Random-TestingResults.cgi?xmlfile=______20090203-0500-Nightly___XML___Test.xml</a><br>
<br> Dart server triggered...<br> Submission successful<br>bill-box:/tmp><br><br><br><br>
<br>_______________________________________________<br>
CMake mailing list<br>
<a href="mailto:CMake@cmake.org">CMake@cmake.org</a><br>
<a href="http://www.cmake.org/mailman/listinfo/cmake" target="_blank">http://www.cmake.org/mailman/listinfo/cmake</a><br></blockquote></div><br></div>