[cmake-developers] Experiments in CMake support for Clang (header & standard) modules

Whisperity whisperity at gmail.com
Mon Oct 15 13:43:07 EDT 2018


Dear List,

(Unfortunately, I can't reply to the previous threads because I just
got to know and subscribed to this list.)

I have (from what it seems, also) created an implementation for C++
Modules TS support (in Clang specifically) for CMake. This is part of
an ongoing research project of mine (and a few of my colleagues) into
the knick and knacks of modular C++ compilation (where "modular" here
is a more abstract term) as we are all harassed by long compilation
times...

The patch is available here:
http://gitlab.kitware.com/cmake/cmake/merge_requests/2482

This talk will specifically be about "Modules TS", *NOT* the "header
modules", or as Richard Smith put it, "C++98 Modules", or the "module
map" thing. (Clang's current documentation really doesn't go into any
detail about the "Modules TS" version sadly...)

(From this point now on, please "s/standard/standard proposal/g" if I
miss it somewhere.)

The idea behind the patch was to introduce a new CMake command
(add_cxx_module) that handles compiling a source file (the module's
input file in the standard) into the necessary module binaries (in
Clang, a PCM - PreCompiled Module), by making sure these compilations
do the necessary parameter setting to the compiler to make a module.
This was introduced through some new variables in CMake's "Compiler
Knowledge" files.

The second change was with "target_link_libraries()" which allowed
specifying a module in the linker list, as modules don't only need the
interface to make sure the semantic analysis can run correctly, but
also the implementation details to generate code in an "end-binary".
(Library or executable or whatever.) This was more like a necessary
frog to swallow as unfortunately, CMake when checking the actual
dependencies (include statements) of a source file, even though I made
it match onto "import" statements too, when an import statement is
read (such as "import foo;") I can't "call back" to CMake's global
generator to find the target named "foo", and if so, set it up as a
dependency (just like how header dependencies easily work without
explicit specification of them). So to make sure dependencies can be
mapped (so changing a module CPP will recompile the PCM and also
relink the end binaries), the user needs to explicitly specify them...
this is admittedly a bad thing, I was thinking of eventually figuring
out maybe a generator expression that can be added to a target through
a command which helps this, but so far what I can see is that the
/child/ CMake instance has no way to know about the global state of
things.

I agree that this thing with listing up all modules needed in
"target_link_libraries()" is the most convoluted part.

The better option would be somehow making sure we can do more (reading
and writing of global state) when an "import" statement is
encountered. In this way, we could automatically at the time of
depchecking a source file:
 - figure out what sort of modules are needed from the list of imports
 - find these modules (somehow? maybe through some logic like the
local folder, parent folder, system folder, maybe a list of modules
added -as modules- (kinda like add_cxx_module() in the knowledge
base), where they are compiled from
 - create their compilation (in a way that the current CXX_COMPILER
does it - hence why I added the extra configuration options saying
"Okay Clang does it this way" and then making room for "But GCC/MSVC
does it the other way")
   - (Sidenote: the fact that GCC wants a callback mechanism which if
I understand it correctly would -want to call the build system- is
really not a good idea I think, because that would introduce coupling
from the compiler back to the build wrangler... maintaining the
knowledge about particular compilers in CMake is already a burden, now
doing a backwards thing in each and every compiler will just turn it
into a big mess of dependency hell.)
   - Of course the whole thing isn't that much clear on Clang's side
anyways. There is a concurrent discussion on the Clang Developers list
(see my post: http://clang-developers.42468.n3.nabble.com/Modules-TS-Work-tt4062259.html#a4062511
and the thread available from here...).
 - add these compilations into the current build, and add their making
as a dependency to build the particular object which imports them, add
them as a linkage dependency (if the compiler, and Clang does, require
such), etc.

This latter step is the cornerstone which requires a particular local
generator to be able to see the global generator -truly globally- (as
current calling FindTarget() on a valid and defined module *even in my
patch!* in depcheck time will result in a nullptr...), and maybe
introduce a "temporarily built things" or "build staging area" kind of
thing where these generated PCMs can be added where they will be read
as valid targets, where they are generated and as files used from, but
which is not -explicitly- mentioned in the original CMakeLists.txt in
any way.

I have yet to try the implementation (and module system as a whole) on
large projects. It is currently in the making on a smaller and a
larger project for research purposes on complexity, memory usage, etc.
I hope the results will be valuable for the whole community.

The whole naming thing is wonky as CMake already has something called
"module library" which is yet another thing, and CMake "Modules" are
just a bunch of CMake files, ...

Regards,
Whisperity.


More information about the cmake-developers mailing list