[cmake-developers] cmake build does too much work

[Matthew Woehlke]

On 2013-12-11 17:40, Bill Hoffman wrote:
> On 12/11/2013 5:13 PM, Matthew Woehlke wrote:
>> I've been working on a project lately that isn't *that* massively large,
>> but has an unusually high number of library and executable targets. One
>> thing that's been bugging me is that any trivial change in a "lower
>> level" library causes more than a hundred targets to be relinked, for no
>> good reason I know of.
>>
>> Now, I *do* get that relinking is good if the library ABI changes.
>> However, that's not the case here, and I am wondering if it would be
>> possible for CMake to generate an additional, intermediary step after
>> library linking to somehow export a file representing the ABI of the
>> library (with overwrite checks to not modify the file if the ABI has not
>> changed), and to use *those*, rather than the actual libraries, as the
>> dependencies for targets linking to the libraries. I think this could
>> produce a significant speed-up for incremental builds in some cases, as
>> it would allow the build to short-circuit the relink of many targets
>> when it turns out a library's ABI has not changed.
>>
>> Does this sound like something CMake could/should do?
>>
>> (I'm thinking something like running objdump on the resulting library
>> with suitable arguments and doing a copy_if_different on the output. I
>> guess this would only apply to shared libraries, and probably should be
>> an optional feature.)
>
> There is this property:
> http://www.cmake.org/cmake/help/git-master/variable/CMAKE_LINK_DEPENDS_NO_SHARED.html

Yes, but that can lead to a broken build if the ABI does change.

(Hi, Stephen. I still oppose making the above on by default due to the 
possibility of broken builds. The difference here is that we are 
actually doing extra work in order to determine if a relink is necessary 
of if it is *safe* to skip it. IOW, I still choose correctness over 
performance; the difference is that the proposed technique would let us 
have both :-).)

> The other option sounds interesting but hard to implement in a cross
> platform manner.

Agreed, but even if it only worked on some platforms it could be a 
significant benefit to them. It's the sort of feature that can just be 
ignored if not supported with no loss in correctness (only performance).

Actually because of its .lib files I wonder if maybe Windows either 
already doesn't have this problem, or at least would be much easier to 
provide this feature there. (I'd be mainly interested in Linux.)

Hmm, come to think of it, if this were to be implemented, I wonder if it 
is faster for the ABI "stamp" to be an actual dump of the ABI, or a 
checksum (e.g. md5 / sha256) of the ABI dump... The latter would trade 
the extra cost computing the checksum for I/O; not needing to write the 
ABI dump to disk at all if it can be read via a pipe, and then only 
needing to read and possibly write a few bytes for the 'stamp' file.

Actually, I suppose the build tool could also do this, but then the 
benefit is only realized by users of that build tool. (On the other 
hand, all users of that build tool would realize the benefit, and not 
just users of CMake.) Still, this now feels worth cross-posting to ninja...

-- 
Matthew