MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0015555CMakeCMakepublic2015-05-05 17:492016-06-10 14:31
ReporterTodd Lipcon 
Assigned ToKitware Robot 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionmoved 
PlatformOSOS Version
Product VersionCMake 2.8.12.2 
Target VersionFixed in Version 
Summary0015555: Ninja: C compilation targets unnecessarily depend on libraries
DescriptionWhen the Ninja generator generates targets for .cc files, it adds dependencies on any libraries that their target depend on. This means that in a parallel build, ninja won't start building any .cc files from one library until all depended-upon libraries have been built -- an unnecessary restriction. The only requirement should be that a target cannot be _linked_ until its required libraries are built.

This severely limits build parallelism in large projects. I manually post-processed a build.ninja file from a medium-size C++ project to remove these dependencies, and a parallel build using icecc on a small cluster was reduced from 1m30s to 1m0s. Looking at a Gantt chart of the build, I could see that all of the cluster cores were kept occupied much better after the fix, especially at the start of the build.
Steps To Reproducetodd@todd-ThinkPad-T540p:/tmp/test$ cat foo.cc
extern int bar();
int main() {
  return bar();
}
todd@todd-ThinkPad-T540p:/tmp/test$ cat bar.cc
int bar() {
  return 1;
}
todd@todd-ThinkPad-T540p:/tmp/test$ cat CMakeLists.txt
add_library(bar bar.cc)
add_executable(foo foo.cc)
target_link_libraries(foo bar)
todd@todd-ThinkPad-T540p:/tmp/test$ cmake -GNinja .
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/test
todd@todd-ThinkPad-T540p:/tmp/test$ grep 'build.*foo.cc.o:' build.ninja
build CMakeFiles/foo.dir/foo.cc.o: CXX_COMPILER foo.cc || libbar.a


I attached the above directory to this bug report.
Additional Information
TagsNo tags attached.
Relationships
related to 0013799closed Kitware Robot Optionally disable build order dependency target_link_libraries 
Attached Filesgz test.tar.gz (804) 2015-05-05 17:49
https://public.kitware.com/Bug/file/5449/test.tar.gz
Issue History
Date ModifiedUsernameFieldChange
2015-05-05 17:49Todd LipconNew Issue
2015-05-05 17:49Todd LipconFile Added: test.tar.gz
2015-05-05 18:03Ben BoeckelNote Added: 0038717
2015-05-05 18:06Ben BoeckelRelationship addedrelated to 0013799
2015-05-05 18:13Todd LipconNote Added: 0038718
2016-06-10 14:29Kitware RobotNote Added: 0042772
2016-06-10 14:29Kitware RobotStatusnew => resolved
2016-06-10 14:29Kitware RobotResolutionopen => moved
2016-06-10 14:29Kitware RobotAssigned To => Kitware Robot
2016-06-10 14:31Kitware RobotStatusresolved => closed

Notes
(0038717)
Ben Boeckel   
2015-05-05 18:03   
This is done because `bar` may have add_custom_command dependencies which are implicitly required by `foo` (such as a generated header). The dependency can be dropped if the transitive non-target dependencies of `bar` are a subset of the explicit non-target dependencies of `foo`. In most cases, both will be the empty set, but this is the safer solution. I believe there is an issue for this task already (it isn't Ninja-specific).
(0038718)
Todd Lipcon   
2015-05-05 18:13   
Yea, in fact with my post-processing I ran into the same issue. Some of the dependended-upon libraries generated headers which were needed by my target.

But, it seems like we could iterate through the dependencies and propagate any "custom command" dependencies directly into "bar", right?
(0042772)
Kitware Robot   
2016-06-10 14:29   
Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.