[cmake-developers] Proposal to teach cmDepends to only take dependencies from the source tree

[Attila Krasznahorkay]

Dear All,

In our software infrastructure I now got to the point that in building "high level" packages/projects the CMake build spends a very noticeable amount of time in calculating the dependencies of source files. To give you a feel of the issue, in one example that I was profiling, I build a small "package" that uses >180 other "packages", which at this point are all installed on a network file system (AFS). Those low level packages themselves depend on N external packages as well (like Boost, Oracle, etc.).

So the include list for the build becomes very long, with most of the needed headers being outside of the source tree. In this case evaluating all the dependencies for a shared library made up of just 3 source files can take >10 seconds.

I tried to find a way to tell CMake to ignore the dependencies that are outside of the source tree. As at this point for my use case those files can be considered fixed. I don't need to consider that they may change without me knowing. But as far as I could see, there is no mechanism in the code at the moment to achieve this.

I did consider teaching cmDepends to only take non-system include paths into account for the dependency calculation. So that whatever is specified as a system include path, would be ignored. But since I found that GCC becomes significantly slower in building my sources if I specify >100 -isystem /afs/... paths to it compared to specifying all those paths with -I, I gave up on this idea.

I also considered adding a new regular expression to cmDependsC to be able to specify using a regular expression which include paths to take into account and which to ignore. Which could still be a good idea to implement in cmDepends (not cmDependsC) later on, but for my own use case something simpler was enough.

In the end I applied the following patch to CMake 3.4.1 locally to speed it up for my use case very significantly. Of course this is not a patch that could be applied to CMake for a general audience. But I do think that if this code/behaviour could be switched on using something like a directory property / global variable, a lot of users could make good use of it. As it can be a reasonable assumption in many development environments that only the changes inside of the source tree should be tracked by the build system.

If you think that this is a good idea, I would be willing to provide some help in implementing it "properly" as well.

Cheers,
             Attila

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cmDepends-sourceOnly.diff
Type: application/octet-stream
Size: 1904 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20151208/c5560900/attachment-0001.obj>