[CMake] Parallel builds do not work correctly when using "cmake -E copy" to copy files

Alan W. Irwin irwin at beluga.phys.uvic.ca
Wed Dec 12 20:10:57 EST 2007 

A set of custom rules to copy files from the source tree to the build tree
is screwing up for parallel builds on Debian testing with cmake 2.4.7. The
parallel builds are done with "make -j 2" on a core duo system (Intel E6550
2.33 MHz).  I am not sure whether the issue is something wrong with our
CMake code, a bug in CMake, or a bug in GNU make, and I would appreciate
your comments to help sort out the possibilities.

Here is the cmake stanza to set up the custom rules and overall custom
target that depends on those rules:

# Copy file and scripts to the binary directory if different to the 
# source directory. Needed for ctest, but also so the tclIndex file
# is generated in the binary tree not the source tree.
if(NOT CMAKE_CURRENT_BINARY_DIR STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
   set(tclIndex_WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
   set(tclIndex_DEPENDS)
   foreach(file ${tcl_SCRIPTS} ${tcl_FILES})
     set(
     tclIndex_DEPENDS
     ${tclIndex_DEPENDS}
     ${CMAKE_CURRENT_BINARY_DIR}/${file}
     )
     add_custom_command(
     OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${file}
     COMMAND ${CMAKE_COMMAND} -E copy
     ${CMAKE_CURRENT_SOURCE_DIR}/${file} ${CMAKE_CURRENT_BINARY_DIR}
     DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/${file}
     )
   endforeach(file ${tcl_SCRIPTS} ${tcl_FILES})
   add_custom_target(tcl_examples ALL DEPENDS ${tclIndex_DEPENDS})
else(NOT CMAKE_CURRENT_BINARY_DIR STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
   set(tclIndex_DEPENDS ${tcl_FILES} ${tcl_SCRIPTS})
   set(tclIndex_WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
endif(NOT CMAKE_CURRENT_BINARY_DIR STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")

For the case of separate build tree, the resulting Makefile stanzas look
like the following :

examples/tcl/CMakeFiles/tclIndex_examples_tcl: examples/tcl/x01
examples/tcl/CMakeFiles/tclIndex_examples_tcl: examples/tcl/x02
...etc.

examples/tcl/x01: /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x01
         $(CMAKE_COMMAND) -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles $(CMAKE_PROGRESS_1)
         @$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --blue --bold "Generating x01"
         cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x01 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl

examples/tcl/x02: /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x02
         $(CMAKE_COMMAND) -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles $(CMAKE_PROGRESS_1)
         @$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --blue --bold "Generating x02"
         cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x02 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
...etc.

This works fine for serial builds (make -j 1), but for parallel builds 
(make -j 2) there is some problem with the interpretation of the
dependencies which means the copy commands are sometimes duplicated.

x01 and x02 were copied correctly, but the x03, x04, and x05 files
were copied twice with the latter double copy causing a severe error.

Here is part of the "make -j 2" output:

make -f examples/tcl/CMakeFiles/tclIndex_examples_tcl.dir/build.make examples/tcl/CMakeFiles/tclIndex_examples_tcl.dir/build
make[2]: Entering directory `/home/software/plplot_cvs/HEAD/build_dir'
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
[ 22%] Generating x01
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x01 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
[ 22%] make[2]: Leaving directory `/home/software/plplot_cvs/HEAD/build_dir'
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles  18
Generating x02
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x02 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
[ 23%] Built target make_documentation
make -f examples/tcl/CMakeFiles/tcl_examples.dir/build.make examples/tcl/CMakeFiles/tcl_examples.dir/build
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
make[2]: Entering directory `/home/software/plplot_cvs/HEAD/build_dir'
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
[ 23%] [ 23%] Generating x03
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x03 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
Generating x03
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x03 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
[ 23%] [ 23%] Generating x04
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x04 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
Generating x04
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x04 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
[ 23%] [ 23%] Generating x05
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x05 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
Generating x05
cd /home/software/plplot_cvs/HEAD/build_dir/examples/tcl && /usr/bin/cmake -E copy /home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x05 /home/software/plplot_cvs/HEAD/build_dir/examples/tcl
Error copying file "/home/software/plplot_cvs/HEAD/plplot_cmake/examples/tcl/x05" to "/home/software/plplot_cvs/HEAD/build_dir/examples/tcl".
make[2]: *** [examples/tcl/x05] Error 1
make[2]: Leaving directory `/home/software/plplot_cvs/HEAD/build_dir'
/usr/bin/cmake -E cmake_progress_report /home/software/plplot_cvs/HEAD/build_dir/CMakeFiles 
make[1]: *** [examples/tcl/CMakeFiles/tclIndex_examples_tcl.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Naively, the GNU Makefile rules generated by CMake seem to look OK, but I
recall from an experience several years ago that it is easy to screw up
dependencies when switching between directories so perhaps the above issues
are due to CMake not being careful enough about that.

Anyhow, I hope there is a CMake and GNU make expert here that can figure out
what the dependency problem is that is causing "make -j 2" to sometimes
execute rules twice with a resulting severe error.

BTW, the "cmake -E" command documents the "copy" subcommand as

copy file destination   - copy file to destination (either file or directory)

so I assume the above rule

COMMAND ${CMAKE_COMMAND} -E copy
${CMAKE_CURRENT_SOURCE_DIR}/${file} ${CMAKE_CURRENT_BINARY_DIR}

would produce identical results (including dependency checking) to

COMMAND ${CMAKE_COMMAND} -E copy
${CMAKE_CURRENT_SOURCE_DIR}/${file} ${CMAKE_CURRENT_BINARY_DIR}/${file}

However, I haven't tried the latter variation yet.

Also, if there is no obvious answer or workaround for this issue, I would be
happy to make a simple example (I guess it would require at least 5 small
files to copy as in this case) which anybody can test on their own single
processor, dual processor, or whatever hardware.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________

More information about the CMake mailing list