[CMake] Compilation with multiple jobs fails

Steven Bellens steven.bellens at mech.kuleuven.be
Mon Sep 20 09:45:19 EDT 2010 

Hi,

I'm trying to compile a custom library using CMake code. The
configuration and compilation goes well when I just use 'make'. In
order to speed things up I tried to compile the library with multiple
jobs in parallel with 'make -j2'. However, compilation always fails in
that case (tested on multiple computers (Fedora - Ubuntu x86_64)). I'm
using cmake 2.8.0. I'v emailed this to the developers of the library
as well (see emails below). The piece of cmake code involved is
indicated below as well, but I'll repeat it here for clarity:

The error I get:
-----------------------------------------------------------
"Generating dyncall-0.3/ConfigVars
* configure package dyncall
guess operating system linux
guess arch x64
guess tool gcc
guess assembler as
[  2%] Generating dyncall/lib/libdyncall_s.a, dyncall/lib/libdynload_s.a
make[3]: Leaving directory
`/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_toolchain_ros/typelib/build'
make[4]: Entering directory
`/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_toolchain_ros/typelib/build/build/orocos-toolchain-typelib/bindings/ruby/dyncall-0.3'
make[3]: Entering directory
`/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_toolchain_ros/typelib/build'
make[4]: *** read jobs pipe: Is a directory.  Stop.
make[4]: *** Waiting for unfinished jobs...."
----------------------------------------------------------------

And the corresponding cmake code:

-----------------------------------------------------------
 ADD_CUSTOM_COMMAND(OUTPUT ${DYNCALL_SOURCE_DIR}/ConfigVars
       COMMAND tar xzf ${CMAKE_CURRENT_SOURCE_DIR}/dyncall-0.3.tar.gz
       COMMAND cd ${DYNCALL_SOURCE_DIR}
&& ./configure --prefix=${DYNCALL_INSTALL_DIR}
       DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/dyncall-0.3.tar.gz)

   ADD_CUSTOM_COMMAND(OUTPUT ${DYNCALL_LIBRARIES}
       COMMAND make "CFLAGS=-O3 -fPIC"
       COMMAND make install
       DEPENDS ${DYNCALL_SOURCE_DIR}/ConfigVars
       WORKING_DIRECTORY ${DYNCALL_SOURCE_DIR})

   ADD_CUSTOM_TARGET(dyncall DEPENDS ${DYNCALL_LIBRARIES})
-----------------------------------------------------------

I'm not that of a cmake expert, but I see that dyncall needs DYNCALL_LIBRARIES,
DYNCALL_LIBRARIES depends on ConfigVars,
and ConfigVars are created with the custom command

Any ideas what goes wrong here?

best regards,

Steven


---------- Forwarded message ----------
From: Leopold Palomo-Avellaneda <leopold.palomo at upc.edu>
Date: 2010/9/15
Subject: Re: [Orocos-Dev] Typelib compilation with multiple jobs fails
To: "orocos-dev at lists.mech.kuleuven.be" <orocos-dev at lists.mech.kuleuven.be>
Cc: Adolfo Rodríguez Tsouroukdissian
<adolfo.rodriguez at pal-robotics.com>, Steven Bellens
<Steven.Bellens at mech.kuleuven.be>


A Dimecres 15 Setembre 2010, Adolfo Rodríguez Tsouroukdissian va escriure:
> On Wed, Sep 15, 2010 at 10:55 AM, Steven Bellens <
>
> steven.bellens at mech.kuleuven.be> wrote:
> > 2010/9/14 Peter Soetens <peter at thesourceworks.com>:
> > > On Tuesday 14 September 2010 11:08:01 Steven Bellens wrote:
> > >> Hi,
> > >>
> > >> compiling the typelib package with multiple jobs fails:
> > >>
> > >> cd build && make -j2
> > >> make[1]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make[2]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make[3]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make[3]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' Scanning dependencies of target dyncall
> > >> Scanning dependencies of target typeLib
> > >> make[3]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make[3]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' [  0%] Generating dyncall-0.3/ConfigVars
> > >> make[3]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make[3]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' [  2%] Building CXX object
> > >> build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/registry
> > >>.o * configure package dyncall
> > >> guess operating system linux
> > >> guess arch x64
> > >> guess tool gcc
> > >> guess assembler as
> > >> [  2%] Generating dyncall/lib/libdyncall_s.a,
> > >> dyncall/lib/libdynload_s.a make[4]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > chain_ros/typelib/build/build/orocos-toolchain-typelib/bindings/ruby/dync
> >al
> >
> > >> l-0.3' make[4]: *** read jobs pipe: Is a directory.  Stop.
> > >> make[4]: *** Waiting for unfinished jobs....
> > >> make[5]: Entering directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > chain_ros/typelib/build/build/orocos-toolchain-typelib/bindings/ruby/dync
> >al
> >
> > >> l-0.3/dyncall' make[5]: *** read jobs pipe: Is a directory.  Stop.
> > >> make[5]: *** Waiting for unfinished jobs....
> > >> make[5]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > chain_ros/typelib/build/build/orocos-toolchain-typelib/bindings/ruby/dync
> >al
> >
> > >> l-0.3/dyncall' make[4]: *** [dyncall] Error 2
> > >> make[4]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > chain_ros/typelib/build/build/orocos-toolchain-typelib/bindings/ruby/dync
> >al
> >
> > >> l-0.3' make[3]: ***
> >
> >
> > [build/orocos-toolchain-typelib/bindings/ruby/dyncall/lib/libdyncall_s.a]
> >
> > >>  Error 2
> > >> make[3]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make[2]: ***
> >
> >
> > [build/orocos-toolchain-typelib/bindings/ruby/CMakeFiles/dyncall.dir/all]
> >
> > >>  Error 2
> > >> make[2]: *** Waiting for unfinished jobs....
> > >> [  4%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/typebuilder
> >.o
> >
> > >> [  6%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/typemodel.o
> >
> > >> [  8%] Building CXX object
> > >> build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/typename
> > >>.o [ 10%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/typevisitor
> >.o
> >
> > >> [ 12%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/typedisplay
> >.o
> >
> > >> [ 14%] Building CXX object
> > >> build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/exporter
> > >>.o [ 16%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/pluginmanag
> >er
> >
> > >> .o [ 18%] Building CXX object
> > >> build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/value.o
> > >> [ 20%] Building CXX object
> > >> build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/importer
> > >>.o [ 22%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/csvoutput.o
> >
> > >> [ 24%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/endianness.
> >o
> >
> > >> [ 26%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/memory_layo
> >ut
> >
> > >> .o [ 28%] Building CXX object
> >
> > build/orocos-toolchain-typelib/typelib/CMakeFiles/typeLib.dir/value_ops.o
> >
> > >> Linking CXX shared library ../../../../lib/libtypeLib.so
> > >> make[3]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' [ 28%] Built target typeLib
> > >> make[2]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make[1]: *** [all] Error 2
> > >> make[1]: Leaving directory
> >
> > `/home/steven/src/svn/cturtle/install/stacks/kul-ros-pkg/stacks/orocos_to
> >ol
> >
> > >> chain_ros/typelib/build' make: *** [all] Error 2
> > >>
> > >> while compiling it with 'make -j1' always goes fine. Any ideas why
> > >> this happens?
> > >
> > > Looks like a bug in the target definition or names in the cmake
> > > files...
> >
> > Don't
> >
> > > have a clue...
> >
> > I've taken a look into the CMake code. The problem is the dyncall
> > extension for ruby. The CMakeLists.txt there contains customs commands
> > to extract a tar archive and subsequently configure and build it. I
> > don't know what exactly happens but the message "make[4]: *** read
> > jobs pipe: Is a directory.  Stop." makes me think it tries to build
> > the code before extraction is completed.
> >
> > Without looking at the code, looks like that there might be a missing
>
> dependency between the untar and build steps. One of the
> add_custom_commands signatures has a DEPENDS option [1]. If what you have
> is targets, you could get away with add_dependencies(buildTarget
> untarTarget) [2]
>

looking on the code, I have found the dependencies in the custom commands.

-----------------------------------------------------------
 ADD_CUSTOM_COMMAND(OUTPUT ${DYNCALL_SOURCE_DIR}/ConfigVars
       COMMAND tar xzf ${CMAKE_CURRENT_SOURCE_DIR}/dyncall-0.3.tar.gz
       COMMAND cd ${DYNCALL_SOURCE_DIR}
&& ./configure --prefix=${DYNCALL_INSTALL_DIR}
       DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/dyncall-0.3.tar.gz)

   ADD_CUSTOM_COMMAND(OUTPUT ${DYNCALL_LIBRARIES}
       COMMAND make "CFLAGS=-O3 -fPIC"
       COMMAND make install
       DEPENDS ${DYNCALL_SOURCE_DIR}/ConfigVars
       WORKING_DIRECTORY ${DYNCALL_SOURCE_DIR})

   ADD_CUSTOM_TARGET(dyncall DEPENDS ${DYNCALL_LIBRARIES})
-----------------------------------------------------------

I understand that to build dyncall needs DYNCALL_LIBRARIES:
- DYNCALL_LIBRARIES depends on ConfigVars
- and ConfigVars are created with the custom command _after_ tar and after the
configure step (configure creates the ConfigVars file)


So, I don't see any clear error here. Maybe is a cmake bug ....

Regards,

Leo

More information about the CMake mailing list