[CMake] Parallel make & custom command

[Alan W. Irwin]

On 2011-06-30 22:15+0200 Alexander Neundorf wrote:

> On Wednesday 29 June 2011, Marcel Loose wrote:
> ...
>> After I had posted my question I realized that this issue has come up
>> quite recently on the mailing list in a thread that I started -- see
>> http://www.mail-archive.com/cmake@cmake.org/msg36362.html. Although the
>> original question was related to building mulitple targets in parallel,
>> Michael Hertling showed that CMake inherently has problems with parallel
>> builds when custom targets/commands come into play.
>
> Really ?
> We use a lot of custom commands in KDE, all the time, everywhere, and people
> build KDE on big build clusters, i.e. very parallel, without problems.
>
> Maybe there are problems when multiple targets depend on the same generated
> file, but I think the "cmake inherently has problems with parallel builds when
> custom targets/commands come into play." is wrong in this very generic way.

I agree with Alex here.  It is always possible, of course, to set up
custom commands and custom targets properly so that parallel builds
work correctly.  The trick is to avoid two or more targets depending
on the same custom command.  So bug 0012311 should just be closed
since that is based on a misunderstanding of how to properly set up
custom commands and targets.

The above rule of thumb is easy to implement for simple build systems,
but for complex build systems it is easy to make file or target
dependency mistakes so that parallel builds do not work correctly.
This is the number-one issue I have had difficulty with for our PLplot
CMake-based build system because that system has lots of custom
commands and a complex web of file and target dependencies that
changes each time we add new build or test features.

To help with the general difficulty of getting dependencies right,
would it be possible to implement a CMake change so that for a given
flag (say --check-dependencies) the cmake application issued warnings
for file and target dependency issues that would cause problems for
parallel builds for a given CMake-based build system?  That would help
a lot to solve this topic that keeps coming up again and again on this
list and also make existing CMake-based build systems much easier to
maintain.

Assuming such a change was implemented, then the standard FAQ response
to this topic would then boil down to "run cmake with the
--check-dependencies flag to find out the dependency issues you have
to solve before you can reliably run a parallel build."

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________