View Issue Details Jump to Notes ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0003658CMakeCMakepublic2006-08-25 09:422008-01-17 09:16
ReporterBrad King 
Assigned ToBrad King 
PrioritylowSeverityfeatureReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0003658: make in subdirectories should not build other dirs
Descriptionhttp://bugs.kde.org/show_bug.cgi?id=132076 [^]

When "make" is run from the top all dependencies in the project should be taken into account. When "make" is run from subdirectories only dependencies in that directory and below should be used.

There are a few steps that are needed for this. The "check build system" step that happens at the beginning when make is run currently checks all dependencies in the whole tree. It has two purposes:
  - Check for changed CMakeLists.txt files and re-run CMake. This can be converted to look only at CMakeLists.txt files in the subdirectory where make was run.
  - Check for broken implicit dependencies and re-scan the dependencies of offending files. This step can be moved to a per-target check instead of a global check.
TagsNo tags attached.
Attached Files

 Relationships

  Notes
(0004743)
Brad King (manager)
2006-08-25 09:43

I'm assigning this bug to myself since I'll be implementing the fixes.
(0004752)
Alex Neundorf (developer)
2006-08-25 18:40

Do you want to change this completely, so that if I enter make in kdelibs/kioslaves/ftp/ it won't compile because it doesn't compile all the rest before ?
Some see this as a feature, some as a bug.
IMO as it is now is at least very correct behaviour.
Make it configurable ?
(...old KDE problem: we're not sure, so let's make it configurable...)

Alex
(0004753)
Brad King (manager)
2006-08-25 20:57

Almost all of the changes needed to make this work are good changes anyway and may still result in faster builds even if we maintain all dependencies.

We could probably add a "local" or "foo/local" target to do the local dependencies only:

(cd kioslaves/ftp && make) # Builds safely
(cd kioslaves/ftp && make local) # Builds quickly
(cd kioslaves/ftp && make sometarget/local) # Builds sometarget using only local dependencies
(0004791)
Brad King (manager)
2006-08-29 09:21

Another note about how to do this:

All implicit dependency scanning can be moved to a per-target check. Implicit dependencies need to be up to date only just before object file rules are evaluated. If we convert the custom command output generation rules to be a dependency of the foo.dir/depend target then all files needed for implicit dependency scanning will be available at dependency scanning time. This way we will not need a list of generated files for the directory available at dependency scanning time because the files will exist. Also the dependency scanning rule can be changed to an "implicit dependency update" rule that makes sure dependencies are up to date either by verifying existing scanning results or doing a new scan.
 
(0004792)
Brad King (manager)
2006-08-29 09:22

Bug 0002389 is related to this bug.
(0004794)
Alex Neundorf (developer)
2006-08-29 12:32

Maybe instead of introducing the "/local" suffix the existing "/fast" suffix could be used ?
Otherwise the user has to decide whether he want sot build safe "make foo" or really fast and really unsafe "make foo/fast" or somewhere in between "make foo/local".
(0004797)
Alex Neundorf (developer)
2006-08-29 15:06

At work we are using cmake on a quite bug project also under cygwin, and there it is magnitudes slower than under linux. While the dependency checking takes approx. 1 second on linux, it takes something like 15 seconds cygwin. Here "make foo/fast" really helps, while only checking the deps of the current target wouldn't change anything for this case I think. So maybe keeping "/local" and "/fast" separate is actually better.
(0004803)
Brad King (manager)
2006-08-29 15:21

Bug 0003348 requests a slightly different level of granularity, but has been given an approximate solution and closed.
(0004813)
Thomas Zander (reporter)
2006-08-29 17:16

Ideally /local would
a) rebuild sources which depend on items that are newer (like #includes).
b) rebuilds only file that can be reached without going to the parent dir. So foo.cc and bar/baz.cc but not ../dir/file.cc

See for more http://bugs.kde.org/show_bug.cgi?id=132076#c4 [^]
(0004819)
Brad King (manager)
2006-08-31 11:12

I don't think it is practical to do the level of analysis necessary to generate makefiles that behave this way. The /local targets will rebuild targets in the current directory or below and all their target-level dependencies inside the subproject. The locations of the source files used by these targets is not relevant.
(0004820)
Brad King (manager)
2006-08-31 11:19

Bug 0002607 requests exclusion/inclusion of specific directories during dependency scanning. If it is implemented you will be able to stop dependency scanning from going outside your directory. However this will be used whether make is started from the top or not and is therefore unsafe.

Basically we have no way to change the set of file-level dependencies used based on where make was started. It is probably possible to implement but would require separate dependency scanning for each make run and a bunch of make variables to switch which results are included in the make process. This is far too complicated.

(0004822)
Brad King (manager)
2006-08-31 13:33

FYI, I'm not saying this pure subproject build framework is a bad idea, I'm just saying it is not practical to do with a Makefile-based build system. Perhaps in the future someone will produce a build system that supports this and then contribute a generator for it.
(0004823)
Brad King (manager)
2006-08-31 14:11

The following changes are a step towards getting rid of the global check-build-system:

/cvsroot/CMake/CMake/Source/cmDependsC.cxx,v <-- cmDependsC.cxx
new revision: 1.26; previous revision: 1.25
/cvsroot/CMake/CMake/Source/cmDependsC.h,v <-- cmDependsC.h
new revision: 1.18; previous revision: 1.17
/cvsroot/CMake/CMake/Source/cmLocalUnixMakefileGenerator3.cxx,v <-- cmLocalUnixMakefileGenerator3.cxx
new revision: 1.170; previous revision: 1.169
/cvsroot/CMake/CMake/Source/cmMakefileTargetGenerator.cxx,v <-- cmMakefileTargetGenerator.cxx
new revision: 1.41; previous revision: 1.40

The commit comment was
"ENH: Make sure all custom command outputs are up to date before scanning dependencies. This avoids the need to pass a list of generated files to the dependency scanning code and to rescan after the files have been generated. Currently there is no notion of implicit dependencies of the custom commands themselves so this design is safe. We only need to make sure implicit dependencies are up to date before the make process for the /build part of a target is executed because only this process loads them."
(0005049)
Cyrille Berger (reporter)
2006-09-26 16:47

I am not sure how it is related to this bug (but as the bug on subproject in the kde bug database refers to this one). but make uninstall only works at the root of the project, and it's very very much lacking in subdirectory, especially in projects with a lot of plugins when you might want to remove specific plugins for testing.
(0005054)
Brad King (manager)
2006-09-27 14:48

make uninstall is a totally different issue. Please file a separate feature request.

(0010168)
Brad King (manager)
2008-01-17 09:16

CMake HEAD in CVS now generates a scalable make system in which the dependency integrity is checked only for targets on which the requested target depends. Inter-target dependencies outside of subdirectories are still maintained but the global check at make time has been removed. This should be acceptably fast and fully safe for subproject building.

 Issue History
Date Modified Username Field Change
2008-01-17 09:16 Brad King Status assigned => closed
2008-01-17 09:16 Brad King Note Added: 0010168
2008-01-17 09:16 Brad King Resolution open => fixed


Copyright © 2000 - 2018 MantisBT Team