[CMake] ctest ques.: capture driver errs

[David Cole]

Looks like the test didn't fail:

<DartMeasurement name="ImageError" type="numeric/double">0</DartMeasurement><DartMeasurement name="BaselineImage" type="text/string">Standard</DartMeasurement><DartMeasurement name="WallTime" type="numeric/double">0.14</DartMeasurement>

...at least not because of ImageError. The value for the ImageError 
measurement is 0. If it failed, it failed because of something else.

Did ctest report that the test failed? What's the output if you run 
ctest -V -R TestCommonGeomUtils2?

You're on the right track...


dean.inglis at camris.ca wrote:

>Hi David,
>
>thanks for your reply.
>
>I have a couple more questions (for anyone) based on
>CMake testing of VTK/Examples/Build/vtkMy,
>since I seem to be generating image test errors with ctest.
>
>In my root CMakeLists.txt, I added the following to do 
>hopefully basic testing, without submitting results to 
>a dashboard: 
>
>#projectroot/CMakeLists.txt
>
>MARK_AS_ADVANCED(BUILD_TESTING)
>IF(BUILD_TESTING)
>  ENABLE_TESTING()
>ENDIF(BUILD_TESTING)
>
>and in CMakeOptions.cmake:
>
>#projectroot/CMakeOptions.cmake
>
>MARK_AS_ADVANCED(BUILD_TESTING)
>IF(BUILD_TESTING)
>  IF(NOT VTKLOCAL_TESTING_TEMPORARY_DIR)
>    MAKE_DIRECTORY(${PROJECT_BINARY_DIR}/Testing/Temporary)
>    SET(VTKLOCAL_TESTING_TEMPORARY_DIR "${PROJECT_BINARY_DIR}/Testing/Temporary" CACHE INTERNAL "Temporary folder for tests")
>  ENDIF(NOT VTKLOCAL_TESTING_TEMPORARY_DIR)
>  ENABLE_TESTING()
>  OPTION(VTK_USE_DISPLAY "Turn this option off and tests will not popup windows" ON)
>  MARK_AS_ADVANCED(VTK_USE_DISPLAY)
>  SET(CXX_TEST_PATH ${EXECUTABLE_OUTPUT_PATH})
>ENDIF(BUILD_TESTING)
>
>finally, in one of my source code subdirs, I have Testing/Cxx/CMakeLists.txt
>along with the .cxx tests:
>
>#projectroot/Common/Testing/Cxx/CMakeLists.txt
>
>SET(KIT LocalCommon)
>
>SET(MyTests
>  TestCommon.cxx
>  TestCommonGeomUtils.cxx
>  TestCommonGeomUtils2.cxx
>)
>
>CREATE_TEST_SOURCELIST(
>  Tests
>  ${KIT}CxxTests.cxx
>  ${MyTests}
>  EXTRA_INCLUDE vtkTestDriver.h
>)
>
>ADD_EXECUTABLE(${KIT}CxxTests ${Tests} )
>TARGET_LINK_LIBRARIES(${KIT}CxxTests vtkLocalCommon vtkCommon vtkFiltering vtkGraphics vtkRendering)
>
>SET (TestsToRun ${Tests})
>REMOVE (TestsToRun ${KIT}CxxTests.cxx)
>
>FOREACH (test ${TestsToRun})
>  GET_FILENAME_COMPONENT(TName ${test} NAME_WE)
>  ADD_TEST(
>    ${TName} 
>    ${CXX_TEST_PATH}/${KIT}CxxTests
>    ${TName}
>    -V ${VTKLOCAL_SOURCE_DIR}/Testing/baseline/${TName}.png
>    -T ${VTKLOCAL_TESTING_TEMPORARY_DIR}
>  )   
>ENDFOREACH (test) 
>
>
>The third .cxx test uses the standard vtk testing tools
>
><snip>
>  // some VTK graphics pipeline here...
>
>  int retVal = vtkRegressionTestImage( renWin );
>  if ( retVal == vtkRegressionTester::DO_INTERACTOR)
>	{
>	iren->Start();
>	}
>  
>  return retVal;
><snip>
>
>running ctest with NO image in the project source tree's Testing/baseline dir generates
>a (correct) .png test image in the project binary build tree under /Testing/Temporary,
>with LastTest.log file containing:
>
><snip>
>3/3 Testing: TestCommonGeomUtils2
>3/3 Test: TestCommonGeomUtils2
>Command: "d:/Developer/Releases/Borland/vtkStatic/bin/LocalCommonCxxTests.exe" "TestCommonGeomUtils2" "-V" "d:/Developer/ISSCode/vtkLocal/Testing/baseline/TestCommonGeomUtils2.png" "-T" "d:/Developer/Releases/Borland/vtkLocalStatic/Testing/Temporary" 
>Directory: d:/Developer/Releases/Borland/vtkLocalStatic/Common/Testing/Cxx
>"TestCommonGeomUtils2" start time: Sep 20 11:45 Eastern Daylight Time
>Output:
>----------------------------------------------------------
><DartMeasurement name="ImageNotFound" type="text/string">d:/Developer/ISSCode/vtkLocal/Testing/baseline/TestCommonGeomUtils2.png</DartMeasurement>
>
><DartMeasurement name="WallTime" type="numeric/double">0.0900002</DartMeasurement>
>
><DartMeasurement name="CPUTime" type="numeric/double">0.1</DartMeasurement>
>
>----------------------------------------------------------
>"TestCommonGeomUtils2" end time: Sep 20 11:45 Eastern Daylight Time
>"TestCommonGeomUtils2" time elapsed: 00:00:00
>----------------------------------------------------------
><snip>
>
>
>If I copy that image as my baseline for the test into the source tree's Testing/baseline dir
>and then run the test, I encounter a failed test with log entry along with a LastTestsFailed.log
>and no .diff.png or .small.png image(s):
>
><snip>
>3/3 Testing: TestCommonGeomUtils2
>3/3 Test: TestCommonGeomUtils2
>Command: "d:/Developer/Releases/Borland/vtkStatic/bin/LocalCommonCxxTests.exe" "TestCommonGeomUtils2" "-V" "d:/Developer/ISSCode/vtkLocal/Testing/baseline/TestCommonGeomUtils2.png" "-T" "d:/Developer/Releases/Borland/vtkLocalStatic/Testing/Temporary" 
>Directory: d:/Developer/Releases/Borland/vtkLocalStatic/Common/Testing/Cxx
>"TestCommonGeomUtils2" start time: Sep 20 11:48 Eastern Daylight Time
>Output:
>----------------------------------------------------------
><DartMeasurement name="ImageError" type="numeric/double">0</DartMeasurement><DartMeasurement name="BaselineImage" type="text/string">Standard</DartMeasurement><DartMeasurement name="WallTime" type="numeric/double">0.14</DartMeasurement>
>
><DartMeasurement name="CPUTime" type="numeric/double">0.14</DartMeasurement>
>
>----------------------------------------------------------
>"TestCommonGeomUtils2" end time: Sep 20 11:48 Eastern Daylight Time
>"TestCommonGeomUtils2" time elapsed: 00:00:00
>----------------------------------------------------------
><snip>
>
>I thought at least that ctest would generate the usual .small and .diff images
>in a failure case.  Also, I don't see why the test should fail since the test
>image and baseline images are identical.
>
>Dean
>
>
>  
>