MantisBT - CMake
View Issue Details
0013889CMakeCMakepublic2013-02-04 06:002016-06-10 14:31
Micha Hergarden 
Kitware Robot 
normalminoralways
closedmoved 
x86_64LinuxUbuntu 12.04
CMake 2.8.10.2 
 
0013889: Ninja: Multiple add_custom_command for the same output file lead to multiple rules being generated for the same file
The cmake ninja generator adds multiple build rules for the same file, when there are multiple add_custom_command supplied in subdirectories. When ninja is run with the generated build.ninja file this leads to the following warning:

ninja: warning: multiple rules generate /tmp/test.c. build will not be correct; continuing anyway

I am using the latest ninja from the git repository:
https://github.com/martine/ninja.git [^]
commit 05736568b9a26599865dd7452e62f70ecdbc9e41
1. Create a CMakeLists.txt file which adds two subdirectories, e.g.:

cmake_minimum_required(VERSION 2.8)
ADD_SUBDIRECTORY(tst1)
ADD_SUBDIRECTORY(tst2)

2. Create two CMakeLists.txt files for the subdirectories, both of which add a custom command for the same output file, and an executable using that file, e.g.:
ADD_CUSTOM_COMMAND(OUTPUT /tmp/test.c COMMAND cmake ARGS -E copy /home/test/test.c /tmp/test.c)
ADD_EXECUTABLE(test1 /tmp/test.c)

3. Run cmake -G Ninja
   You can now examine the build.ninja file to see the multiple build rules
4. Run ninja
   You will now see the ninja warning.
linux, ninja
zip testproject.zip (3,321,724) 2014-01-28 02:26
https://public.kitware.com/Bug/file/5053/testproject.zip
Issue History
2013-02-04 06:00Micha HergardenNew Issue
2013-02-05 07:04Micha HergardenTag Attached: linux
2013-02-05 07:04Micha HergardenTag Attached: ninja
2013-02-09 06:52Peter KuemmelSummaryMultiple add_custom_command for the same output file lead to multiple rules being generated for the same file => Ninja: Multiple add_custom_command for the same output file lead to multiple rules being generated for the same file
2013-03-09 06:29Peter KuemmelNote Added: 0032556
2013-03-09 06:29Peter KuemmelStatusnew => backlog
2014-01-27 10:35Andreas MohrNote Added: 0034997
2014-01-27 13:57Micha HergardenNote Added: 0035006
2014-01-27 15:42Andreas MohrNote Added: 0035008
2014-01-28 02:26Micha HergardenFile Added: testproject.zip
2014-01-28 02:55Micha HergardenNote Added: 0035009
2014-01-28 03:20Micha HergardenNote Added: 0035010
2016-06-10 14:28Kitware RobotNote Added: 0042216
2016-06-10 14:28Kitware RobotStatusbacklog => resolved
2016-06-10 14:28Kitware RobotResolutionopen => moved
2016-06-10 14:28Kitware RobotAssigned To => Kitware Robot
2016-06-10 14:31Kitware RobotStatusresolved => closed

Notes
(0032556)
Peter Kuemmel   
2013-03-09 06:29   
Bug is not new any more, move to backlog for later processing.
(0034997)
Andreas Mohr   
2014-01-27 10:35   
Hmm, is there any reason that such handling for a fully identical output file ought to be valid?
To me it seems a bit pathological to generate two identical locations from separate parts of the build tree.


Side note: I'm replying to this issue since I just had a somewhat related case (same error) where it turned out that Ninja was in fact rightfully entitled to complain ;)
[I had a custom command producing an output file, and due to ""issues"" I then *manually* set the GENERATED property on that output file (a stupid HACK, in hindsight!), while being blissfully unaware that the output file path had a *slight* mismatch - a "./" part difference stemming from a sometimes missing ${CMAKE_CFG_INTDIR} expression, thus *of course* not being able to match up the produced output file location with the set of input files of the binary target, and thus CMake generating an extra "Assume dependencies for generated source file." section. - "ass-u-me" being the keyword here...]
Got enough keywords now in here that other people ought to be able to find this troubleshooting issue :)

Have to say that Ninja seems quite a bit more useful/strict in build dependency diagnosis/issues than a make-based build :)
(0035006)
Micha Hergarden   
2014-01-27 13:57   
Well, the case above is of course a bit contrived. I get this behaviour when using I am using a module I wrote to wrap ExternalProject_Add for some autoconf/automake projects I have.

In this module I call configure_file to generate some wrappers to be used in the call to ExternalProject_Add. I use these wrappers in the *_COMMAND of the ExternalProject_Add.

One such call looks like:
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/../wrappers/buildwrapper.in ${CMAKE_CURRENT_BINARY_DIR}/buildwrapper @ONLY)

When I configure this project using Ninja, I get the warning for the buildwrapper.in file.

I am willing to change my module of you have alternative suggenstions, but this seems like a valid use of configure_file to me.
(0035008)
Andreas Mohr   
2014-01-27 15:42   
I'm having issues recognizing where exactly the parallel handling is taking place.
So, I currently assume your setup to be something like this:

Super project, which does a configure_file() and then populates an ExternalProject_Add() section with various buildwrapper-using lines (for BUILD_COMMAND etc.).

And you've got various sub directories which then are responsible for instantiating one autoconf-/automake-based project each.


OK, anyway, let's analyze your original test case again:
Given parallel execution it's not deterministic when exactly generation of the output file name (test.c) will happen. And that might just be in the middle of test.c having been generated and now being used for the resulting build step (depender). --> Re-written file --> build breakage, of an ongoing build making use of the first-time generated file.
That, AFAIWT, is exactly why Ninja is complaining.
So, for your contrived test case here, I'd think the unfriendly warning is quite understandable.

For your configure_file() case, I'm a bit astonished:
you say that it's the configure_.in_ file that it is complaining about.
Yet that file is a single *source*-side file, then multiplied into various differing(!?) output locations, via repeated configure_file() execution.
I'm astonished why this would be a problem. Especially since I'd assume the configure_file() call to be configure-time execution, i.e. within the main layer (or are we talking build-time-based sub project here already as well??), where Ninja build-time handling would not even enter the picture yet.

Are you sure that the configure-time configure_file() is the problem, or rather some ensuing build-time use of the source/destination file? (try deactivating build-time use of these to see whether the problem vanishes).
I've had multiple "Ninja issues" already, and every - single - time - I ended up realizing having made some bl00dy build hierarchy/chain precision mistake which the make-based build would not even care about in the least.
(0035009)
Micha Hergarden   
2014-01-28 02:55   
I've uploaded a stripped project which shows the layout I use, and also reproduces this 'issue'. I don't use the superproject setup (yet).

I've tried setting the GENERATED property to False on the *.in file, but that didn't have any effect.

Is there an alternative way to have the common template files?
(0035010)
Micha Hergarden   
2014-01-28 03:20   
Ah.. Your comments got me off in the right direction: you are absolutely right!

By defining the source path for the template I used the cmake_current_source_dir as a known starting point. This resulted in multiple rules for the input template, each having a different path but ultimately resulting in the same path when resolved.

I can remove the warning by making sure the paths for the templates are the same for each project (by using the cmake_source_dir as starting point for instance).

This issue may be closed immediately. Thank you for your time & sorry for the noise.
(0042216)
Kitware Robot   
2016-06-10 14:28   
Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.