[cmake-developers] Idea for Multi-Toolchain Support

Wed Dec 19 14:16:12 EST 2018

Hello,

just chiming in here. I work for a company that works with embedded
devices, so cross compiling is our bread and butter.

We have a super build system - in a combination of CMake and Makefile -
that first builds a certain set of projects for the host platform and
second builds for the actual target platform.
For testing purposes we can also build all projects for the host
platform to run tests.
All of the projects are compiled through ExternalProject, as it's a mix
of open source packages which can use arbitrary build systems and our
own projects, which use CMake.

Within our own projects we already have a set of functions that wrap
add_executable() and add_library() to make our lives easier and enforce
common idioms so adding some multiple toolchains there would be easy.
The open source packages would be a bit trickier as we'd have to ensure
that with the ExternalProject calls it would use the correct toolchain
given a CMake toolchain definition that would then have to apply to a
non CMake build system such as automake.

However, we don't just have one target platform, we have more than 5
combined ARM and MIPS platforms all with different gcc versions, etc,
along with the host build. Each different platform may have a different
set of projects enabled. This depends on a great number of variables,
like hardware support, etc., as well as non technical reasons.

If there was multiple toolchain support, we could squash the super build
system away which gives a bit more of an integrated system - great - but
the complexity of selecting the correct projects to build given a
certain platform would then have to be put somewhere else - not so great.

While I dislike the super build system concept, it does provide for an
abstraction of that complexity for some of these problems.

As we have adopted CMake wider and wider over the past few months, some
of the limitations have become clearer too, mainly the fact it runs on a
single thread.
We have over 400 individual projects with nearly 1000 CMakeLists.txt
files with a total of 27k lines of CMake.
I have performed some experiments to partially convert the super build
system to be purely CMake for the single stages, which was an
interesting exercise.
I found that the process from parsing the CMake files to generating the
50MB worth of build.ninja file took over 30 seconds on an i7 CPU with
SATA SSDs.
If this was further extended to include both host and target that would
result in an even slower build file generation.

The super build system concept helps us here too as by splitting the
parsing of CMakeLists.txt up with ExternalProject the overall generation
of the build.ninja file takes a "mere" ~7 seconds.

This limitation alone means that doing any of this could be really
detrimental to the iterative build cycle and would severely limit the
usability of this feature for us.

    W

On 19/12/2018 05:05, Peter Mitrano wrote:
> [External email]
>
> Hey all, just thought I would chime in and note that I very often write
> code that I expect to run both on my robots hardware and on my
> development machine in a simulator. In this case, I want the same target
> names and everything -- I simply use two cmake build directories, and
> now there is even support for this workflow in CLion so my life is very
> easy. Doing all of those as two targets with distinct names would be
> cumbersome, and the current system works very well for me.
>
> On Tue, Dec 18, 2018 at 8:54 PM frodak17 <frodak17 at gmail.com
> <mailto:frodak17 at gmail.com>> wrote:
>
>
>
>     On Tue, Dec 18, 2018 at 4:41 PM Kyle Edwards
>     <kyle.edwards at kitware.com <mailto:kyle.edwards at kitware.com>> wrote:
>
>         On Tue, 2018-12-18 at 16:12 -0500, frodak17 wrote:
>>         What about conflicting build types when building a host tool
>>         target vs the cross compiler target?
>>         For instance the host tool may (should?) be built as a release
>>         project (default optimization levels) but the cross compiler
>>         project could be built as a debug project (so you can debug it
>>         with no optimizations)?
>>         I guess it would then be CMAKE_<toolchain>_BUILD_TYPE and then
>>         targets using that toolchain would be built with that build type.
>
>         I think that both host and cross targets would be built with the
>         same build type. If you're using a multi-config generator like
>         VS or Xcode, how do you specify which combination of configs you
>         want? It's simpler if every target is built as the same type,
>         whether it's host or cross. Think about it - the current system
>         already expects that every target in the project is built as the
>         same build type. There's no way to specify "foo is built as
>         debug, bar is built as release." Why would this assumption
>         change just because we add a multi-toolchain mechanic?
>
>
>     I have thought about it which is why I asked.  The original
>     assumption of one toolchain per CMake project is being extended. So
>     why not think of extending the build type.  Sure it's simpler to
>     ignore the build type. However, it doesn't make sense to me to build
>     and run a host tool in debug mode when it could be running faster if
>     built in release mode.  This is independent of whatever I'm doing
>     with cross-compiler targets. You're already updating the generator
>     to pick and choose the proper flags just on the tool chain type and
>     it already picks the correct flags based on the configuration type.
>     But instead of the generator looking at the global build type it
>     looks at the target build type. The target build type could default
>     to the global build type and be overridden via a property.
>
>     Multi config generators would be problematic if they don't have a
>     Configuration Manager that allows for setting different projects
>     with different configurations (like Visual Studio) but is that
>     reason enough to exclude the idea for single configuration generators?
>
>>         On the other hand this doesn't really help anyone working with
>>         mixed build systems.  What you can't do in a straight forward
>>         manner is host tool development with Visual Studio or Eclipse,
>>         execute it and use it's results with a cross compiler (that
>>         uses Ninja or Makefiles) targeting an embedded system.
>
>         Every target would be built by the same generator regardless of
>         whether it's a host or cross target. So if you select the VS
>         generator, both the host tools and the embedded software would
>         be built in VS (if this is possible.) Likewise for Ninja and Make.
>
>>         In this case I think that what you want is to easily control
>>         multiple projects using different build systems, the
>>         inter-dependencies, and easily specify that external project A
>>         produces a file for external project B.  Ideally you would be
>>         in the root of the build folder and type 'cmake --build .' and
>>         have it take care of everything with minimal extraneous build
>>         tool executions or sometimes even invoking the build tool in
>>         parallel for external projects that don't depend on each other.
>
>         If you want to build different parts of the software with
>         different generators (build systems) then you're better off
>         using a superbuild with ExternalProject. The intent of this
>         proposal is to allow users to build both host tools and
>         cross-compiled software in the same build tree with the same
>         generator.
>
>
>     Sure.  I was thinking / commenting about the other real world
>     projects that are more complicated.  Can add_custom_command() be
>     used to run a binary that was built as an ExternalProject which it
>     was linked via depends to generate a file that is used by several
>     other ExternalProjects and it's all glued together with proper
>     dependencies so it can be invoked by a single command?
>
>     If this can already be simply done why is anyone asking to use
>     multiple toolchains in a single project?  You just use an
>     ExternalProject with each toolchain file and you're done, or is this
>     a shortcut method instead of using ExternalProject to build the host
>     tool?
>
>     If ExternalProject doesn't define a target that can be used as a
>     dependency for add_custom_command() or add_dependencies() then isn't
>     this a problem which is driving people to ask to use multiple
>     toolchains?
>
>     https://cmake.org/pipermail/cmake-developers/2018-December/030920.html
>
>     This email seems to indicate that ExternalProject can't be used with
>     add_custom_command().
>     --
>
>     Powered by www.kitware.com <http://www.kitware.com>
>
>     Please keep messages on-topic and check the CMake FAQ at:
>     http://www.cmake.org/Wiki/CMake_FAQ
>
>     Kitware offers various services to support the CMake community. For
>     more information on each offering, please visit:
>
>     CMake Support: http://cmake.org/cmake/help/support.html
>     CMake Consulting: http://cmake.org/cmake/help/consulting.html
>     CMake Training Courses: http://cmake.org/cmake/help/training.html
>
>     Visit other Kitware open-source projects at
>     http://www.kitware.com/opensource/opensource.html
>
>     Follow this link to subscribe/unsubscribe:
>     https://cmake.org/mailman/listinfo/cmake-developers
>
>
>
> [External email. Treat hyperlinks and attachments with caution]
>
>
This transmission contains information that may be confidential and contain personal views which are not necessarily those of YouView TV Ltd. YouView TV Ltd (Co No:7308805) is a limited liability company registered in England and Wales with its registered address at YouView TV Ltd, 3rd Floor, 10 Lower Thames Street, London, EC3R 6YT. For details see our web site at http://www.youview.com