View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0015555CMakeCMakepublic2015-05-05 17:492016-06-10 14:31
ReporterTodd Lipcon 
Assigned ToKitware Robot 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionmoved 
PlatformOSOS Version
Product VersionCMake 2.8.12.2 
Target VersionFixed in Version 
Summary0015555: Ninja: C compilation targets unnecessarily depend on libraries
DescriptionWhen the Ninja generator generates targets for .cc files, it adds dependencies on any libraries that their target depend on. This means that in a parallel build, ninja won't start building any .cc files from one library until all depended-upon libraries have been built -- an unnecessary restriction. The only requirement should be that a target cannot be _linked_ until its required libraries are built.

This severely limits build parallelism in large projects. I manually post-processed a build.ninja file from a medium-size C++ project to remove these dependencies, and a parallel build using icecc on a small cluster was reduced from 1m30s to 1m0s. Looking at a Gantt chart of the build, I could see that all of the cluster cores were kept occupied much better after the fix, especially at the start of the build.
Steps To Reproducetodd@todd-ThinkPad-T540p:/tmp/test$ cat foo.cc
extern int bar();
int main() {
  return bar();
}
todd@todd-ThinkPad-T540p:/tmp/test$ cat bar.cc
int bar() {
  return 1;
}
todd@todd-ThinkPad-T540p:/tmp/test$ cat CMakeLists.txt
add_library(bar bar.cc)
add_executable(foo foo.cc)
target_link_libraries(foo bar)
todd@todd-ThinkPad-T540p:/tmp/test$ cmake -GNinja .
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/test
todd@todd-ThinkPad-T540p:/tmp/test$ grep 'build.*foo.cc.o:' build.ninja
build CMakeFiles/foo.dir/foo.cc.o: CXX_COMPILER foo.cc || libbar.a


I attached the above directory to this bug report.
TagsNo tags attached.
Attached Filesgz file icon test.tar.gz [^] (804 bytes) 2015-05-05 17:49

 Relationships
related to 0013799closedKitware Robot Optionally disable build order dependency target_link_libraries 

  Notes
(0038717)
Ben Boeckel (developer)
2015-05-05 18:03

 This is done because `bar` may have add_custom_command dependencies which are implicitly required by `foo` (such as a generated header). The dependency can be dropped if the transitive non-target dependencies of `bar` are a subset of the explicit non-target dependencies of `foo`. In most cases, both will be the empty set, but this is the safer solution. I believe there is an issue for this task already (it isn't Ninja-specific).
(0038718)
Todd Lipcon (reporter)
2015-05-05 18:13

 Yea, in fact with my post-processing I ran into the same issue. Some of the dependended-upon libraries generated headers which were needed by my target.

But, it seems like we could iterate through the dependencies and propagate any "custom command" dependencies directly into "bar", right?
(0042772)
Kitware Robot (administrator)
2016-06-10 14:29

 Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

 Issue History
Date Modified Username Field Change
2015-05-05 17:49 Todd Lipcon New Issue
2015-05-05 17:49 Todd Lipcon File Added: test.tar.gz
2015-05-05 18:03 Ben Boeckel Note Added: 0038717
2015-05-05 18:06 Ben Boeckel Relationship added related to 0013799
2015-05-05 18:13 Todd Lipcon Note Added: 0038718
2016-06-10 14:29 Kitware Robot Note Added: 0042772
2016-06-10 14:29 Kitware Robot Status new => resolved
2016-06-10 14:29 Kitware Robot Resolution open => moved
2016-06-10 14:29 Kitware Robot Assigned To => Kitware Robot
2016-06-10 14:31 Kitware Robot Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team