[CMake] Possible Bug when using ctest to build/test debug and release in same binary tree

[Miguel A. Figueroa-Villanueva]

Hello all,

I think I found a bug with ctest and below I report what I have found so 
far and my hack at it in the mean time. I will appreciate clarifications 
from the more experienced CMake users, and I would gladly help 
reproduce/debug/fix this if it is in fact a bug.

First of all, I'm not an expert CMake user... so, I apologize up front 
if I'm misunderstanding things here. Second, this is related to my 
earlier thread: "problems submitting nightly build with ctest for 
debug/release".


Goal
----
I am interested in using ctest script to build/test/submit a nightly 
build of VXL (and CMake as well, now that I'm at it), but I want to have 
the debug and release builds in the same binary tree (although the 
source tree is separated).


System
------
- OS: WinXP Pro with SP2
- Compiler: Visual Studio 7 .NET 2003
- Machine: AMD Athlon XP 2400+ 2GHz 1GB DDR RAM
- cmake and vxl updated daily from cvs


Problem/Bug
-----------
After several hours (well hours throughout several days!! ;) ), I can't 
get the debug configuration built with the following (typed in the 
command line):

cmake /path/to/source/CMake
ctest -D NightlyStart
ctest -D NightlyConfigure
ctest -C Debug -D NightlyBuild

By inspecting the output files, the build command is:

.../devenv.com CMake.sln /build Release /project ALL_BUILD

ctest -C Debug -D NightlyTest -- does choose the correct tests to build, 
only they all fail because they weren't built. Something like this:

""
CMake Error: config type specified on the command line, but test 
executable not found.D:/builds/CMake/bin/Debug/ctest 50/ 50 Testing 
VSExternalInclude

The following tests FAILED:
           1 - kwsys.testSystemTools (Not Run)
           2 - kwsys.testProcess-1 (Not Run)
                       ...
          50 - VSExternalInclude (Not Run)
""


Diagnostics
-----------
After going into the ctest source code and using the debugger, I see 
that it is using the command read from the DartConfiguration.tcl file 
and it is not overiding the "/build" value with the "-C" flag. That is, 
it doesn't modify the makecommand in none of:

m_CTestConfiguration["MakeCommand"]
m_CTestConfigurationOverwrites["MakeCommand"]

from cmCTest class in Source/cmCTest.cxx

I guess that when the "-C" flag is encountered, it should set something 
somewhere and at some point generate the make command according to this 
value (maybe in one of the two map variables above). However, I don't 
understand the CMake/CTest architecture enough to know where (or if) it 
should be setting it.


Work-Around
-----------
The only way I have been able to get around this is by supplying the 
MAKECOMMAND variable in the cmake command as in:

cmake -DMAKECOMMAND:STRING=".../devenv.com CMake.sln /build Debug
       /project ALL_BUILD" /path/to/source/CMake
ctest -D NightlyStart
ctest -D NightlyConfigure
ctest -C Debug -D NightlyBuild // I think I don't need the -C Debug here
ctest -C Debug -D NightlyTest  // I do need it here

I don't know if this is the intended behavior, but it is unintuitive to 
me that if I can supply the "-C Debug" in the ctest command line it only 
works for selecting the correct test, but not building them... Sorry for 
the long blabering, if for other reasons it is designed this way.


Finally
-------
My solution is being tested and submitted to dashboard under:

site: miguelf.home_desktop

and you can see the script file used in the Notes.xml file.


Thanks in advance for any help you can provide me,
--Miguel