[cmake-developers] Optionally disabling static lib dependencies with COMPILE_DEPENDS

Tue Jan 9 05:30:05 EST 2018

Perhaps keep an eye on the work going on in merge request 1524
<https://gitlab.kitware.com/cmake/cmake/merge_requests/1524>, which may
allow you to use object libraries in target_link_libraries(). Not sure if
it would cut the dependencies in the way you are seeking, but maybe take a
look and see.

On Fri, Jan 5, 2018 at 8:08 PM, John Wordsworth <johnwordsworth at gmail.com>
wrote:

> First of all - thanks for the suggestions. I intend to look into ccache
> for our Linux builds in the future and there are some useful tips here for
> me to look into. I wasn't aware of clcache and it looks very interesting,
> especially combined with network sharing.
>
> I realise I wasn't very clear in my previous explanation. Hopefully this
> paints a better picture;
>
> - We have 40 or so developers spread across Visual Studio 2015 (MSBuild -
> Windows), Xcode (macOS) and Make/Clang (Linux). 80% use Windows.
> - We have a core set of libs in use by a number of projects (~30 static
> libs and ~10 different projects), but the projects mix and match which libs
> they use - merging the static libs (beyond a few of them) isn't something
> we want to do.
> - I have recently been revising our CMake structure and have boiled it
> down to a fairly simple "add_library(x), target_link_libraries(x,
> default_compile_settings, dependencies)" which means compile settings,
> include directories and macros propagate through the libs nicely.
> - We sometimes want to build and use certain libs as shared libs instead
> of static libs.
>
> With regards to the COMPILE_DEPENDS feature that I have mocked up and
> started testing with our team;
>
> - If and only if COMPILE_DEPENDS is set for a STATIC library, then when
> building the target dependencies for that library, it uses the list of
> provided targets instead of those that would have been inferred from
> previous calls to "target_link_libraries" or "add_dependencies".
>
> The reality is, the libs which are being built statically can nearly all
> build in parallel with only one or two "real" dependencies. I understand
> that our case is rather specific, but having implemented "COMPILE_DEPENDS"
> and written ~10 lines of CMake in our project (hard coding the one or two
> actual build order dependencies and disabling the rest), our compile graph
> looks much nicer and saves a significant amount of time building on a
> single machine (see https://www.johnwordsworth.com/temp/cmake_compile_
> depends.jpg). We see similar savings on macOS using Xcode but have not
> tested on Linux yet. When we allow Incredibuild to distribute building of
> compilation units across 15-20 agents, the proportional build time drops
> more dramatically between the two (the same project goes from ~2.6mins ->
> ~1.6mins).
>
> I understand that just "hard overriding" the target dependencies is a bit
> messy, but I'd also be happy to explore other ways I could add a similar
> feature to CMake if there is a potentially better way to do this. What I am
> ideally looking for in a solution is that I can continue to propagate
> include directories / compile settings / preprocessor macros through the
> chain of libs and that it improves building static libs in parallel across
> all our build systems (MSVC / XCode / make). Ideally, I wouldn't have to
> restructure my CMake project too much either - as the draft I have for our
> new structure feels super clean now and refactoring it just to remove build
> order dependencies would be shame. If there are any alternative ideas for
> how to implement this, I'd love to discuss. I'm no CMake expert, but maybe
> if we could use "target_link_libraries" with OBJECT libraries to grab
> compile settings, or perhaps I could look into a mechanism for not adding
> build order dependencies on VS2015/Xcode if there is no need too?
>
> Thanks again for the feedback and discussion so far.
>
> // John W
>
> On Thu, Jan 4, 2018 at 12:16 PM, Eric Noulard <eric.noulard at gmail.com>
> wrote:
>
>>
>> 2018-01-04 10:48 GMT+01:00 Craig Scott <craig.scott at crascit.com>:
>>
>>>
>>>
>>> On Thu, Jan 4, 2018 at 8:27 AM, John Wordsworth <john at johnwordsworth.com
>>> > wrote:
>>>
>>>> I have recently been reviewing ways to improve build times for our
>>>> project, which is comprised of a number of static libraries. I stumbled
>>>> across this post on the CMake tracker from 2012/13 (
>>>> https://cmake.org/Bug/view.php?id=13799). It suggests adding a
>>>> COMPILE_DEPENDS target property to to explicitly set dependencies for
>>>> STATIC libraries instead of always using all linked libraries as
>>>> build-order dependencies.
>>>>
>>>> Having done a draft implementation in a local CMake repository it has
>>>> shaved off  20% of our 120s build time. I expect the savings to be much
>>>> more dramatic when I test with Incredibuild (approximately 50% based on
>>>> tests done previously from just deleting dependencies manually in Visual
>>>> Studio).
>>>>
>>>
>> You said you tested with Incredibuild but with what kind of configuration?
>> AFAIK Incredibuild takes its power from distributing the build? So does
>> your CMake test uses something that distribute the build too ?
>>
>>
>>>
>>>> I don’t really want to refactor our code to use “OBJECT” libraries as
>>>> the inability to link with other targets means that propagating compile
>>>> options / include directories etc down the chain of linked libs becomes
>>>> painful. This method allows me to switch between static and shared libs
>>>> using a config option and none of my CMake scripts need to change.
>>>>
>>>
>>> There's a couple more choices here. If your project consists of lots of
>>> small (static) libraries, consider whether you can combine some of them to
>>> result in a smaller number of larger libraries. This isn't always a gain,
>>> but in terms of ability to compile sources in parallel, it will often lead
>>> to more efficient builds. You just need to be careful you don't end up with
>>> so many objects being combined into one library that you start to hit max
>>> open file limits during linking/archiving (something I've hit on multiple
>>> platforms lately, so it's not just a hypothetical example). Use of
>>> target_sources() can be quite helpful if you want to try out this path (you
>>> may find this article
>>> <https://crascit.com/2016/01/31/enhanced-source-file-handling-with-target_sources/>
>>> helpful).
>>>
>>> Another choice is to go in the opposite direction and take advantage of
>>> the optimisation made for the Ninja generator (if that's a choice open to
>>> you) that was introduced in CMake 3.9.0
>>> <https://gitlab.kitware.com/cmake/cmake/merge_requests/430> where if no
>>> custom commands exist on a target A, then compilation steps of another
>>> target B linking to A are allowed to proceed without waiting for A's link
>>> step to complete. Only B's link step will depend on A's link step. In your
>>> project, if you have custom commands, see if you can split up that target
>>> into just those sources that need the results of the custom command and
>>> another target that doesn't. The latter will then be able to compile
>>> earlier, so fewer sources have to wait for earlier linking steps. This
>>> might be hard to do, it really depends on how your project is structured.
>>>
>>
>> I fully agree with Craig here. We use CMake + Ninja and we get really
>> efficient parallel build (on 10+ cores machines).
>> This is on Linux though but it seems MS is putting effort to get CMake +
>> ninja build work with Visual Studio:
>> https://blogs.msdn.microsoft.com/vcblog/2017/05/10/cmake-sup
>> port-in-visual-studio-whats-new-in-2017-15-3-update/
>>
>> Another way to improve your build may be to review your dependency graph
>> carefully. You can use cmake --grahviz options to dump the graph and see
>> whether you can cut
>> some dependencies due to transitivity of dependency. I have seen many
>> projects with "overspecified deps". I think this impair build time when
>> using a lot of static libs because in that case you may get
>> poor performance link time. Namelly if you have many unit tests linking
>> to too many static libs because of overspecified deps.
>>
>> That said you did not specify whether if the 120s build time is only for
>> the library or if this is a "global" build time which includes linking
>> somes executables and possible unit tests programs.
>>
>> Note that even if this is not the case having a "big-picture" look at the
>> dependency build graph generated by CMake may be insightful.
>> Note that with CMake 3.10 you can now see difference between PUBLIC,
>> PRIVATE and INTERFACE deps:
>> https://cmake.org/cmake/help/v3.10/release/3.10.html
>>
>>
>>
>>>
>>> Both of the above choices allow you to retain the automatic propagation
>>> of compile options, include directories, etc. and to switch between
>>> shared/static easily, but the latter is specific to the Ninja generator and
>>> may not be an acceptable change for you.
>>>
>>>
>>>>
>>>> Anyway, I was wondering whether there was any interest in me pushing my
>>>> solution back to Git / submitting a Pull request so that it might be merged
>>>> in at some point. If there is - any advice on any gotchas I might watch for
>>>> instead of just adding some fairly simple code to
>>>> cmComputeTargetDepends.cxx would be gratefully received - especially as
>>>> this is my first time poking around in CMake code.
>>>>
>>>
>>> The existing behaviour is conservative and any change would have to also
>>> be conservative, meaning that it must not introduce any possibility of
>>> breaking existing projects. If I'm understanding your proposed feature
>>> correctly, it sounds like you want to relax the build-order dependencies by
>>> default when a COMPILE_DEPENDS target property is defined. Basically, if
>>> COMPILE_DEPENDS is defined, you are taking over responsibility for the
>>> build-order dependencies. This would be something I'd usually discourage
>>> projects from doing because such manual dependencies would be a prime
>>> candidate for not being kept up to date as a project evolves, leading to
>>> subtle, hard-to-trace build errors. Some judicious project restructuring
>>> can normally give a pretty efficient parallel build without having to
>>> resort to such measures, so I'm wary of adding a feature like this (though
>>> I can understand the desire for it).
>>>
>>> In my experience, you can get some considerable speedups using tools
>>> like ccache (and its equivalents for other platforms/compilers). These
>>> obviously only help for subsequent builds of things that have been built
>>> previously, but for everyday development where you switch between branches
>>> or for CI servers doing lots of similar builds, the savings can be
>>> impressively big.
>>>
>>
>> +1 again.
>>
>> ccache boost our build. It can be more than 10x faster (depends on cache
>> hit).
>> You can even share (on a local network) the ccache directory between
>> developers and the CI and get very high hit rate (> 80%).
>>
>> Now I'm not working on windows nor with Visual Studio and AFAIK ccache
>> does not work with MSVC.
>> There seem to be alternative, https://github.com/frerich/clcache, but as
>> you guessed I did never used that.
>>
>> --
>> Eric
>>
>
>

-- 
Craig Scott
Melbourne, Australia
https://crascit.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake-developers/attachments/20180109/f53dae2b/attachment-0001.html>