MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0008536CMakeCMakepublic2009-02-15 04:102016-06-10 14:30
ReporterNick Fortino 
Assigned ToBrad King 
PrioritylowSeverityfeatureReproducibilityalways
StatusclosedResolutionmoved 
PlatformOSOS Version
Product VersionCMake-2-6 
Target VersionFixed in Version 
Summary0008536: shared object dependency more strict than it could be
DescriptionLet me preface this by stating that fixing this is probably more trouble than it's worth, but I thought I'd report it anyway:

when using TARGET_LINK_LIBRARIES on a library in the project, cmake causes the specified executable to depend on the build for that library. Thus, before any work is done, the library build must finish. If the user didn't specifically specify the dependency directly, it should be safe to assume the user only wants the library build for the library itself, and therefore only needs the resulting .so file at link time. Thus, cmake should generate a make file that when run with 'make -j' builds the sources both for the library and the executable in parallel, and then waits for the library build to finish before linking the executable.
Steps To Reproduce
Additional InformationThis would have great build time benefits in cases where the project contains a long building core library, and many program files which depend on the core library, as the program files can still do most of the work in parallel, leaving only the relatively quick link step for the end.

I'm presuming a cmake dev would know better how one might implement this, but the makefile structure could do this if the root makefile had two targets for each build target, one for compiling and one for linking.
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2009-02-15 04:10Nick FortinoNew Issue
2009-02-18 17:04Bill HoffmanStatusnew => assigned
2009-02-18 17:04Bill HoffmanAssigned To => Brad King
2009-02-18 17:38Brad KingNote Added: 0015188
2009-02-18 17:39Brad KingPrioritynormal => low
2009-02-18 17:39Brad KingSeveritytweak => feature
2009-02-18 17:39Brad KingResolutionopen => suspended
2009-02-20 19:03Nick FortinoNote Added: 0015281
2012-08-13 10:36Brad KingStatusassigned => backlog
2012-08-13 10:36Brad KingNote Added: 0030496
2016-06-10 14:27Kitware RobotNote Added: 0041502
2016-06-10 14:27Kitware RobotStatusbacklog => resolved
2016-06-10 14:27Kitware RobotResolutionsuspended => moved
2016-06-10 14:30Kitware RobotStatusresolved => closed

Notes
(0015188)
Brad King   
2009-02-18 17:38   
There are already two main makefile targets per build target, one for implicit dependency computation, and one for building/linking. You're correct that we could split the building/linking further. However, doing that split while avoiding asking make to re-check many dependencies will require the generated build.make files to be split into two pieces too.

Currently if a library's build rules have custom commands to generate files users can assume that these rules will have run by the time an executable that links to the library starts to build. Making this change would require projects to explicitly add inter-target dependencies for this case. While this breaks backwards compatibility, the CMake Policy mechanism would let us keep old projects working while allowing the behavior for new projects.

My main concern about this feature is how it affects generated header files. Consider the case where the library's build rules generate a header file to be included by the executable's sources. In order to safely link a target to this library one must use both target_link_libraries and add_dependencies. However, the Visual Studio generators will not require the add_dependencies part so projects written by a VS user could have subtle Makefile bugs.

I conclude that this behavior is more trouble than it's worth, especially if it is the default.

In your example use case, if the library is so large then all the parallel build job tokens should be consumed building its object files for most of the time. The only wasted time is while the last few objects are compiling and the library is linking and preventing the executables' objects from starting. Once the link finishes then all the executables can build in parallel. Can you provide example timings with a manually hacked makefile on a real project that show a significant benefit?
(0015281)
Nick Fortino   
2009-02-20 19:03   
My example case does, in fact, come from a real project, which I have been re-writing the make system for:
http://meshlab.sourceforge.net/ [^]

It turns out I will be sticking with qmake, due to it's better support for precompiled headers, as it has the relatively odd feature that the headers take ~3min to compile, and then everything else is instant if precompiled headers are used (otherwise the build takes 10 hours).

I freely admit this case is very esoteric, and I don't expect the feature to be implemented for my personal project, but I'll record it here in case similar situations are more common. Here are timings for building the core library and program in series and in parallel (Note, I have yet to commit the changes to the project. At the time of writing, this can't be independently verified. The results are very predictable, however, given the information that all the time is spent building the headers twice: once with CFLAGS for a shared object, and once for a program)

Series:
time make -j2
real 6m23.716s
user 6m49.240s
sys 0m5.480s

Parallel:
time make -j2
real 3m27.456s
user 6m48.700s
sys 0m5.670s
(0030496)
Brad King   
2012-08-13 10:36   
Sending issues I'm not actively working on to the backlog to await someone with time for them.

If an issue you care about is sent to the backlog when you feel it should have been addressed in a different manner, please bring it up on the CMake mailing list for discussion. Sign up for the mailing list here, if you're not already on it:

 http://www.cmake.org/mailman/listinfo/cmake [^]

It's easy to re-activate a bug here if you can find a CMake developer or contributor who has the bandwidth to take it on.
(0041502)
Kitware Robot   
2016-06-10 14:27   
Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.