<blockquote style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;" class="gmail_quote">Thanks very much. It's maybe more than can/should be done using
"ctest". That is, some shell scripting might be in order as well...<br></blockquote><div> </div>Well, I can tell you that all of what you want and more are completely possible using CMake/CTest alone, because I'm doing everything you list below already. :)<br>
<br>I'm going to slightly re-order some of your requests below, just FYI.<br><br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I have a CDash server set up. Every night, shortly after midnight, I want a bunch of build machines, running various operating systems, to build and test the software, and report their results to CDash.<br>
<br></blockquote><div><br>Yep, I'm doing this, with the added excitement that the various OS and test-suite configurations are brought up on Amazon Elastic Compute Cloud virtual machines. (Automating that was not as easy as I would have liked, I can tell you.)<br>
<br><br><blockquote style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;" class="gmail_quote">If possible, I'd like to ensure that when ctest invokes cmake, the
CMake variables mentioned above end up with the same values as they
would have if the shell scripts mentioned above had been used to kick
of cmake. This is a relatively low-priority requirement, but it would
be nice.<br></blockquote><div><br>What I think you need to accomplish all of what you want is to create a "new-style" ctest script. When used in this way, ctest really is a full scripting language, with the added benefits of being fully cross-platform and integrated with cmake.<br>
<br>IMHO, this is amongst the very best features of cmake/ctest. It's unfortunately also one of the least well-documented. The best documentation I'm aware of at the moment is this page:<br><br> <a href="http://www.cmake.org/Wiki/CTest:Using_CTEST_and_CDASH_without_CMAKE">http://www.cmake.org/Wiki/CTest:Using_CTEST_and_CDASH_without_CMAKE</a><br>
<br>which is not somewhere it would ever occur to you to look because you ARE using CMake, but there it is. If you ignore the stuff about CTestTestfile.cmake (which cmake will generate for you) and the callouts to execute_process(), it covers most of what you need to know.<br>
<br>To start with, create a baseline ctest script, which explicitly does the same steps that ctest -D does implicitly. It will look something like this:<br><br>set (CTEST_SOURCE_DIRECTORY "/path/to/sourcecode")<br>
set (CTEST_BINARY_DIRECTORY "${CTEST_SOURCE_DIRECTORY}/build")<br>ctest_start (Nightly)<br>ctest_update (SOURCE "${CTEST_SOURCE_DIRECTORY}")<br>ctest_configure (BUILD "${CTEST_BINARY_DIRECTORY}")<br>
ctest_build(BUILD "${CTEST_BINARY_DIRECTORY}")<br>ctest_test(BUILD "${CTEST_BINARY_DIRECTORY}")<br>ctest_submit ()<br><br>(Note: this depends on the source code already having been checked out from svn. The URL I listed above has some instructions about configuring it to do the initial checkout automatically, but that's something to add a bit later when the basics are working.)<br>
<br></div><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
My project consists of two, somewhat-related bodies of source code: MOOS and IVP. Each has its own CMake-based build system. Each comes with a shell script that invokes "cmake" with some options, and then "make". The options supplied to cmake like CMAKE_BUILD_TYPE, CMAKE_CXX_FLAGS_RELEASE, and some application-specific CMake variables.<br>
<br></blockquote><div><br>You should be able to specify those variables in your ctest script, before any of the ctest_*() commands are used. If the flags and variables are different in various configurations, then I'd suggest the following: First, create simple .cmake files with a complete set of variables in it. For instance, create Debug-linux.cmake with<br>
<br> set (CMAKE_BUILD_TYPE "Debug")<br> set (CMAKE_CXX_FLAGS_DEBUG "...whatever...")<br><br>Put all of those .cmake files in a subdirectory next to your ctest script called "conf", and then put the following at the top of your ctest script:<br>
<br>include ("${CTEST_SCRIPT_DIRECTORY}/conf/${CTEST_SCRIPT_ARG}") <br></div><div><br>Now, to run the "Debug-linux" configuration, you can invoke your script (from cron or whatever) as<br><br> ctest -S /path/to/script.cmake,Debug-linux<br>
<br>(Aside: this is the only way I know to pass variables to a ctest script; -D doesn't work because it's used for Dashboards. Basically, whatever you put after a comma in the -S argument to ctest will be made available to the ctest script as CTEST_SCRIPT_ARG. It's ugly, but it gets the job done. Oh, and CTEST_SCRIPT_DIRECTORY is set to the directory of the ctest script itself.)<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
To date I've been ignoring MOOS with regards to CTest. I do have to build MOOS prior to building IVP, but I wanted to keep things simple for now and just focus on ctest'ing IVP.<br>
<br>
Since MOOS and IVP are in separate source trees and are built at separate times, I have no expectation that their building / testing results can end up in the same CDash dashboard. But it would be awfully nice if they did, because we really see them as two parts of the same product. It's only an accident of nature that they're not part of the very same CMake build tree.<br>
</blockquote><div><br>There's no reason that two different products can't submit to the same CDash project. Just specify the same CDash URL in your CTestConfig.cmake.<br><br>When you've gotten things working the way you want, you may want to read this:<br>
<br> <a href="http://www.kitware.com/blog/home/post/11" target="_blank">CDash subprojects...</a>
<br><br>and the Kitware Quarterly newsletter that is mentioned therein. That will explain how to make CDash display things more nicely for multiple projects by designating them as subprojects of a main project. But that is an optional extra. <br>
</div><div> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Furthermore, I want a shell command that I've designed to decide, for each build, what appears in CDash's "Build Name" column. I want this because "Linux-c++" doesn't sufficiently differentiate the various build platforms I'm using.<br>
</blockquote><div><br>This is easily done via the CTEST_BUILD_NAME variable, which I see has been discussed in another thread. Hopefully once you get a functional ctest script going, this will Just Work(tm) for you now. Just put it as another set() line in your ctest script or the included configuration file.<br>
</div><div> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
If possible, I'd like the CDash report to include a mention of how the sourcecode checkout went. If the sourcecode checkout fails, I'd like CDash to show that, rather than just having no report from the client that had the problem.<br>
</blockquote><div> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I want a fresh checkout of the source code each time I do this. Merely updating it isn't clean enough. Also, I'm using svn.<br></blockquote><div><br>The URL I listed above had some information about how to get ctest to do an initial svn checkout for you. I personally am explicitly invoking "svn co" with execute_process(), which means that any failures there don't end up on cdash as you say. I think possibly if you do what the URL lists, though, you might get what you want.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I'd like to continue specifying my tests by having "ADD_TEST(...)" calls sprinkled throughout my project's CMakeLists.txt files.<br></blockquote><br></div>Yes, please; don't ever stop doing that. It's just the right approach.<br>
<br>Hope this helps! Good luck... as I said, you can make ctest do some wonderful things. My current script is much more involved than this, and it handles a remote build/test server for our code as well as nightly testing on several platforms with different build and test-suite configurations.<br>
<br>Ceej<br>aka Chris Hillery<br>