[cmake-developers] Using multiple toolchains

[Stephen Kelly]

Brad King wrote:

> On 7/1/2013 1:26 PM, Stephen Kelly wrote:
>> I've pushed a proof-of-concept multiple-toolchains branch to my clone
>> which enables the use of multiple toolchains with cmake.
> 
> Wow, that is a mind-blowing branch ;)

That's a good reaction :).

>> The idea is to use a separate cache and set of definition overrides for
>> each toolchain.
> 
> That sounds a lot like a separate invocation of CMake for each
> toolchain, which is what one must do now.

It's not very similar. 

* There is less for the buildsystem maintainer or invoker of the cmake 
executable to do.
* The cmake-targets for the host and the target are 'closer together'. There 
doesn't need to be a separate buildsystem of host tools.

>> One of the goals I have is to make it possible to use N different
>> toolchains, not just two.
> 
> Currently we support only one toolchain per language per build tree,
> not even two.  When one needs host tools we require two separate
> builds.

Correct. Or ExternalProject as you say below.

>> There's obviously a clash of output binary names, cmake target names and
>> make target names if the same library is going to be build for multiple
>> toolchains. That's something I have not yet attempted to solve.
> 
> These among many other challenges await anyone that attempts your
> proposed approach.

... but only if the goal is N toolchains. 

My approach comprises several steps:

1. Refactor cmake so that more that one toolchain can be available at 
   a time.
2. Make host+target builds possible in a single buildsystem by initializing 
   both the host and the target toolchain, specifying whether to find 
   dependencies in the host or the target, and specifying whether to build a 
   particular cmake-target for the host or for the target system. This does 
   not expose us to problems of clashes of cmake-target names for multiple 
   toolchains or problems of defining multiple per-toolchain make-targets 
   for a single cmake-target as all of those things remain impossible at
   this step. This step requires some way to define such distinctions in 
   front-end CMakeLists.txt files. Possibly something like a toolchains() 
   command scoped to end with a endtoolchains() command.
3. Make it possible to define multiple toolchains to build a single target 
   for. Still only two toolchains are possible (host+target), but now we can
   define that a single cmake-target created with add_executable should be 
   built for both the host and the target. Consider moc for example, which 
   might make sense to build for both so that target-on-target builds (or 
   target-in-target-vm builds) can be done. In addition to step two, this 
   step requires solving the disabmiguation problems of cmake-targets and 
   other problems that come from using multiple toolchains from one
   cmake-target definition.
4. Make it possible to use N target toolchains. This takes advantage of the 
   solutions created in step 3. In addition to step 3, this step requires 
   deprecation of the undocumented CMAKE_TOOLCHAIN_FILE in favor of 
   something which can be a list, and it requires some way of attaching 
   names to toolchains. Projects using this have the disadvantage that there 
   is nothing but convention to standardise toolchain names. One project 
   might use RaspPi as a toolchain name in a CMakeLists.txt file, while 
   another uses RaspberryPi and yet another Raspbian. Qt has a similar issue 
   with device mkspec names.

> While the work in your branch has gotten
> impressively far, it also serves to demonstrate the inherent
> complexity of the proposed approach.  IMO it is not worth exploring
> that approach further.  Sorry.

That reaction is not what I was hoping for. :)

My branch makes a start mostly on steps 1 and 2 above, and to a small 
extent, step 4. 80% of the motivation and value of multiple toolchains comes 
from step 2, so I would happily truncate the other two as goals or leave 
them to future exporation.

I don't think an inappropriate amount of work or fundamental change to cmake 
is required to achieve step 2.

> Much of the multiple toolchains functionality can be accomplished by
> using ExternalProject to create a "superbuild" in which each inner
> project uses a different toolchain file.  From that one can get to
> a single make invocation to drive everything.  It doesn't need any
> fundamental changes in CMake.

I haven't used ExternalProject before today, though I was already aware of 
the existence of it, having read various discussions about it on the mailing 
lists. I added an example to my branch which uses it in an equivalent way 
with a simple host+target buildsystem.

Apart from learning how to use it, which I documented with commits, I have 
the following notes:

* The host compilations can be run in parallel, and the target compilations 
  can be run in parallel, but the host compilations can't be done in 
  parallel with the target compilations. A larger project than my example 
  would have compilations which do not depend on files generated by the 
  host_tool. This limitation is introduced by using ExternalProject but 
  would not be present in an implementation of my step 2.

* I wonder how the parallelization issue raised by Alan is handled: 

   http://thread.gmane.org/gmane.comp.programming.tools.cmake.user/47103/focus=47109

  I build with make flags defined in my environment and having -jN 
  cause Nxnum_parallel_ExternalProjects jobs to be run instead of N is a bit 
  odd. That would not happen with my step 2.

* I need to be explicit about the dependencies between ExternalProjects, 
  instead of cmake figuring it out for me, as would happen with my step 2.

* The separation between the host and the target ExternalProjects is 
  unfortunate, and should not really be necessary.

* I needed to hardcode a relative path to the host build dir in order to get 
  to the exported targets file. It looks like it would be possible to use 
  ExternalProject_Get_Property to create another connection between the 
  host and target at the superbuild layer. This should not really be 
  necessary.

* The superbuild layer is an additional maintenance layer for the 
  maintainer and something sure to grow in complexity over time.

* With increasing complexity, there is sure to be more data transfer issues 
  and issues of escaping etc to think about by passing though the additional 
  cmake superbuild layer.

* Even if I'm not cross compiling, I have all of the above issues to deal 
  with. This just increases the barrier to cross-compiling, which is not
  necessary.


ExternalProject is a rather large sledgehammer, and while I'm sure it's 
useful and fully appropriate in many cases, in this case it seems to be 
quite inelegant. In this case, we have only two elements in the superbuild, 
both use cmake, neither need to be downloaded, they have strong connections 
to each other, and the only need for separation comes from 
building the targets with separate toolchains.

Implementing my step 2 above is a far more elegant solution, doesn't go far 
enough to require difficult and significant redesign of cmake, and would be 
quite a killer feature for a cross-platform, cross compiling tool. At KDAB 
we do an increasing amount of embedded development and having a feature like 
this would make it easier to convince customers to use cmake instead of 
qmake+many kludges, which we usually have to deal with.

Thanks,

Steve.