[cmake-developers] Experiments in CMake support for Clang (header & standard) modules

[Stephen Kelly]

David Blaikie wrote:

> (just CC'ing you Richard in case you want to read my ramblings/spot any
> inaccuracies, etc)
> 
> Excuse the delay - coming back to this a bit now. Though the varying
> opinions on what modules will take to integrate with build system still
> weighs on me a bit

Can you explain what you mean by 'weighs on you'? Does that mean you see it 
as tricky now? 

I've kind of been assuming that people generally think it is not tricky, and 
I'm just wrong in thinking it is and I'll eventually see how it is all 
manageable.

> - but I'm trying to find small ways/concrete steps to
> make some progress on this rather than being lost in choice/opinion
> paralysis.

Cool.

> To that end, Stephen, I've made a fork of your example repository & a very
> simple/direct change to use C++ modules as currently implemented in Clang.

Interesting, thanks for doing that! Here's the link for anyone else 
interested:

 https://github.com/dwblaikie/ModulesExperiments

> The build.sh script shows the commands required to build it (though I
> haven't checked the exact fmodule-file dependencies to check that they're
> all necessary, etc) - and with current Clang top-of-tree it does build and
> run the example dinnerparty program.

Ok. I tried with my several-weeks-old checkout and it failed on the first 
command with -modules-ts in it (for AbstractFruit.cppm - the simplest one).

I'll update my build and try again, but that will take some time.

> If you happen to try experimenting with any ways the commands in the
> build.sh file could be run from CMake it a sensible way - even if you
> hypothesize what -MM support (or other compiler hooks like the dependency
> server I alluded to above, etc) for modules might look like to do so, I'd
> love to chat about it/throw ideas around/try mocking up/prototyping the
> sort of compiler support (I don't think there's -MM support yet, but I
> could see about adding it, for example) that seems like it might be most
> useful.

Yes, that would be useful I think.

> One thing I'm vaguely concerned about is actually the dependency of
> building modules within a single library (as in this project/example - at
> least the way I've built it for now - I didn't try building it as separate
> .so/.a files). At least across-library we can work at the library
> granularity and provide on the command line (or via a file as GCC does)
> the module files for all the modules from dependent libraries. But I'm not
> sure how best to determine the order in which to build files within a
> library

Exactly, that's one of the reasons my repo has libraries of multiple files 
with dependencies between them, but building everything into one executable 
also exposes that issue.

I guess it would work in a similar way to determining the order to link 
libraries though, so probably not a major problem.

> - that's where the sort of -MM-esque stuff, etc, would be
> necessary.

Would it? I thought the -MM stuff would mostly be necessary for determining 
when to rebuild? Don't we need to determine the build order before the first 
build of anything? The -MM stuff doesn't help that.

>> It seems quite common to have one PCH file per shared library (that's
>> what Qt does for example). What makes you so sure that won't be the case
>> with modules?
>>
> 
> Can't say I've worked with code using existing PCH - if that seems common
> enough, it might be a good analogy/guidance people might follow with C++
> modules.

Perhaps. I wonder how the C++ code/build.sh code would change in that 
scenario? If there were only four modules - one for each of the libraries 
(as delimited in the CMakeLists). Would the C++ code change then too 
(something about building partial modules from each of the multiple cppm 
files?), or how would the build.sh code change?

>> Ok. That's not much better though. It still means editing/generating the
>> buildsystem each time you add an import.
> 
> 
> Isn't that true today with headers, though? 

No. Imagine you implemented FruitBowl.cpp in revision 1 such that it did not 
#include Grape.h and it did not add the Grape to the bowl.

Then you edit FruitBowl.cpp to #include Grape.h and add the Grape to the 
bowl. Because Grape.h and Apple.h are in the same directory (which you 
already have a -Ipath/to/headers for in your buildsystem), in this (today) 
scenario, you don't have to edit the buildsystem.

In your port, you would have to add an import of Grape (fine, equivalent), 
add the Grape to the bowl (the same as today), but additionally, you have to 

 * add -fmodule-file=Grape.pcm to the compile line or run your buildsystem 
generator such as CMake to cause that compile line to be generated with the 
argument added.
 * Generate Grape.pcm (because the library has 1000 fruit classes in it and 
your buildsystem is smart enough to lazily generate pcm files as needed)

Partly that is a result of the totally-granular approach you took to 
creating modules (one module per class). If you used the approach of one 
module per library, then you would not need to touch the buildsystem. I hope 
that's clear now, as I've mentioned it multiple times, but I wonder if that 
part of what I say is being understood.

> But today the build system
> does this under the covers with header scanning, -MM modes, etc?

Perhaps. I notice that running CMake on my llvm/clang/clang-tools-extra 
checkout takes a non-zero amount of time, and for other buildsystems takes a 
significantly non-zero amount of time.

Many buildsystem generators already avoid the time/complexity of 
automatically regenerating the buildsystem when needed. Users have to leave 
their IDE and run a script on the command line.

I wonder if people will use C++ modules if CMake/their generator has to be 
re-run (automatically or through explicit user action) every time they add 
'import foo;' to their C++ code... What do you think?

> What would be different about having a similar requirement for modules
> (but a somewhat different discovery process)? 

I hope the above is clear. Please let me know if not. Maybe I'm missing 
something still.

Thanks,

Stephen.