<div dir="ltr">Forwarding to the developers list, since my original reply bounced as I wasn't subscribed to the list yet.<br><br><div class="gmail_quote"><div dir="ltr">---------- Forwarded message ---------<br>From: David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>><br>Date: Wed, May 9, 2018 at 3:23 PM<br>Subject: Re: Experiments in CMake support for Clang (header & standard) modules<br>To: Stephen Kelly <<a href="mailto:steveire@gmail.com">steveire@gmail.com</a>><br>Cc: <a href="mailto:cmake-developers@cmake.org">cmake-developers@cmake.org</a> <<a href="mailto:cmake-developers@cmake.org">cmake-developers@cmake.org</a>>, Richard Smith <<a href="mailto:richard@metafoo.co.uk">richard@metafoo.co.uk</a>>, <<a href="mailto:nathan@acm.org">nathan@acm.org</a>><br></div><br><br><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Mon, May 7, 2018 at 9:01 AM Stephen Kelly <<a href="mailto:steveire@gmail.com" target="_blank">steveire@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_2066748816796752644m_-8026624506218320738moz-cite-prefix"><br>
I think this discussion is more suited to the cmake-developers
mailing list. Moving there. </div></div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>Sure thing - I was sort of hoping much/all of this functionality could be implemented at the user-level without changes to cmake itself at least initially - then upstream convenience mechanisms so everyone doesn't have to rewrite the functionality locally.<br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><div class="m_2066748816796752644m_-8026624506218320738moz-cite-prefix">Hopefully Brad or someone else can
provide other input from research already done.</div></div><div text="#000000" bgcolor="#FFFFFF"><div class="m_2066748816796752644m_-8026624506218320738moz-cite-prefix"><br>
<br>
On 05/07/2018 12:49 AM, David Blaikie wrote:<br>
</div></div><div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr"><br>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">The basic commands required are:<br>
<br>
clang++ -fmodules -xc++ -Xclang -emit-module
-Xclang -fmodules-codegen -fmodule-name=foo
foo.modulemap -o foo.pcm<br>
clang++ -fmodules -c -fmodule-file=foo.pcm
use.cpp<br>
clang++ -c foo.pcm<br>
clang++ foo.o use.o -o a.out<br>
</div>
</blockquote>
<br>
</div>
<div text="#000000" bgcolor="#FFFFFF"> Ok.
Fundamentally, I am suspicious of having to have a
-fmodule-file=foo.pcm for every 'import foo' in each
cpp file. I shouldn't have to manually add that each
time I add a new import to my cpp file. Even if it
can be automated (eg by CMake), I shouldn't have to
have my buildsystem be regenerated each time I add
an import to my cpp file either.<br>
<br>
That's something I mentioned in the google groups
post I made which you linked to. How will that work
when using Qt or any other library?<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
- My understanding/feeling is that this would be
similar to how a user has to change their link command
when they pick up a new dependency.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Perhaps it would be interesting to get an idea of how often users
need to change their buildsystems because of a new link dependency,
and how often users add includes to existing c++ files.<br></div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>Right - my mental model was more that modules would be more one-to-one with libraries, rather than the way headers are these days (with many headers for one library). That seems to be incorrect & it's likely there may be multiple modules (perhaps somewhat fewer than headers) for a single library.<br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">I expect you'll find the latter to be a far bigger number. <br>
<br>
I also expect that expecting users to edit their buildsystem, or
allow it to be regenerated every time they add/remove includes would
lead to less adoption of modules. I can see people trying them and
then giving up in frustration. <br>
<br>
I think I read somewhere that the buildsystem in google already
requires included '.h' files to be listed explicitly in the
buildsystem, so it's no change in workflow there. </div></blockquote><div><br></div></div></div><div dir="ltr"><div class="gmail_quote"><div>Not for the users of the library - but yes, a library needs to declare which headers it contains/provides. (I'm not sure it's at the point where that's required across the codebase - but it's a direction things are moving in).</div></div></div><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">For other teams,
that would be a change which could be a change in workflow and
something rebelled against.<br>
<br>
By the way, do you have any idea how much modules adoption would be
needed to constitute "success"? Is there a goal there?</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>I don't know of any assessment like that.<br></div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_quote"><div>Nope, scratch that ^ I had thought that was the
case, but talking more with Richard Smith it seems
there's an expectation that modules will be somewhere
between header and library granularity (obviously some
small libraries today have one or only a few headers,
some (like Qt) have many - maybe those on the Qt end
might have slightly fewer modules than the have
headers - but still several modules to one library
most likely, by the sounds of it)<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Why? Richard maybe you can answer that? These are the kinds of
things I was trying to get answers to in the previous post to iso
sg2 in the google group. I didn't get an answer as definitive as
this, so maybe you can share the reason behind such a definitive
answer?</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>It's more that the functionality will allow this & just judging by how people do things today (existing header granularity partly motivated by the cost of headers that doesn't apply to modules), how they're likely to do things in the future (I personally would guess people will probably try to just port their headers to modules - and a few places where there are circular dependencies in headers or the like they might glob them up into one module).<br></div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>Now, admittedly, external dependencies are a little
more complicated than internal (within a single
project consisting of multiple libraries) - which is
why I'd like to focus a bit on the simpler internal
case first.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Fair enough.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div> </div>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">Today, a
beginner can find a random C++ book, type in a code
example from chapter one and put `g++
-I/opt/book_examples prog1.cpp` into a terminal and
get something compiling and running. With modules,
they'll potentially have to pass a whole list of
module files too.<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
Yeah, there's some talk of supporting a mode that
doesn't explicitly build/use modules in the
filesystem, but only in memory for the purpose of
preserving the isolation semantics of modules. This
would be used in simple direct-compilation cases like
this. Such a library might need a configuration file
or similar the compiler can parse to discover the
parameters (warning flags, define flags, whatever
else) needed to build the BMI.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Perhaps. I'd be interested in how far into the book such a system
would take a beginner. Maybe that's fine, I don't know. Such a
system might not help with code in stack overflow questions/answers
though, which would probably be simpler sticking with includes (eg
for Qt/boost). </div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>I imagine the expectation is that eventually there will just be modules APIs to some libraries/classes, so it won't always be a user choice. (if there's sufficient penetration of modules, as you say - if there's sufficiently ubiquitous build support, etc - then library vendors will only provide a module API without a "backwards compatible" header based API)<br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"> Library authors will presumably have some say, or try to introduce
some 'best practice' for users to follow. And such best practice
will be different for each library.</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>I would expect so - as they do with documentation today. Though if they only provide a modules or only provide a header API - users will use whatever they're provided.<br></div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite">
<div dir="ltr"><div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">I raised some of
these issues a few years ago regarding the clang
implementation with files named exactly
module.modulemap:<br>
<br>
<a class="m_2066748816796752644m_-8026624506218320738m_4089600057544350869m_2648820049216965386m_1970699153108696964m_-801848360278511963moz-txt-link-freetext" href="http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946.html" target="_blank">http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946.html</a><br>
<br>
<a class="m_2066748816796752644m_-8026624506218320738m_4089600057544350869m_2648820049216965386m_1970699153108696964m_-801848360278511963moz-txt-link-freetext" href="http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946i20.html" target="_blank">http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946i20.html</a><br>
<br>
Interestingly, GCC is taking a directory-centric
approach in the driver (-fmodule-path=<dir>)
as opposed to the 'add a file to your compile line
for each import' that Clang and MSVC are taking:<br>
<br>
<a class="m_2066748816796752644m_-8026624506218320738m_4089600057544350869m_2648820049216965386m_1970699153108696964m_-801848360278511963moz-txt-link-freetext" href="http://gcc.gnu.org/wiki/cxx-modules" target="_blank">http://gcc.gnu.org/wiki/cxx-modules</a><br>
<br>
Why is Clang not doing a directory-centric
driver-interface? It seems to obviously solve
problems. I wonder if modules can be a success
without coordination between major compiler and
buildsystem developers. That's why I made the git
repo - to help work on something more concrete to
see how things scale.<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
'We' (myself & other Clang developers) are/will be
talking to GCC folks to try to get consistency here,
in one direction or another (maybe some 3rd direction
different from Clang or LLVM's). As you noted in a
follow-up, there is a directory-based flag in Clang
now, added by Boris as he's been working through
adding modules support to Build2.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
I just looked through the commits from Boris, and it seems he made
some changes relating to -fmodule-file=. That still presupposes that
all (transitively) used module files are specified on the command
line. <br></div></blockquote><div><br></div></div></div><div dir="ltr"><div class="gmail_quote"><div>Actually I believe the need is only the immediate dependencies - at least with Clang's implementation.</div></div></div><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">I was talking about the -fprebuilt-module-path option added by
Manman Ren in <a class="m_2066748816796752644m_-8026624506218320738moz-txt-link-freetext" href="https://reviews.llvm.org/D23125" target="_blank">https://reviews.llvm.org/D23125</a> because that actually
relieves the user/buildsystem of maintaining a list of all used
modules (I hope).</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>*nod* & as you say, GCC has something similar. Though the build system probably wants to know about the used modules to do dependency analysis & rebuilding correctly. So that's something I'm still trying to figure out - if the dependency information (& indeed, the names of the BMI files) can be wholely known only by the compiler or wholely <br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">Having just read
all of my old posts again, I still worry things like
this will hinder modules 'too much' to be
successful. The more (small) barriers exist, the
less chance of success. If modules aren't
successful, then they'll become a poisoned chalice
and no one will be able to work on fixing them.
That's actually exactly what I expect to happen, but
I also still hope I'm just missing something :). I
really want to see a committee document from the
people working on modules which actually explores
the problems and barriers to adoption and concludes
with 'none of those things matter'. I think it's
fixable, but I haven't seen anyone interested enough
to fix the problems (or even to find out what they
are).<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
Indeed - hence my desire to talk through these things,
get some practical experience, document them to the
committee in perhaps a less-ranty, more concrete form
along with pros/cons/unknowns/etc to hopefully find
some consistency, maybe write up a document of "this
is how we expect build systems to integrate with this
C++ feature", etc.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Great. Nathan Sidwell already wrote a paper which is clearer than I
am on some of the problems:<br>
<br>
<a class="m_2066748816796752644m_-8026624506218320738moz-txt-link-freetext" href="http://open-std.org/JTC1/SC22/WG21/docs/papers/2017/p0778r0.pdf" target="_blank">http://open-std.org/JTC1/SC22/WG21/docs/papers/2017/p0778r0.pdf</a><br>
<br>
However he told me it 'wasn't popular'. I don't know if he means the
problems were dismissed, or his proposed solution was dismissed as
not popular.<br>
<br>
Nevertheless, I recommend reading the problems stated there.</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>Yeah, thanks for the link - useful to read.<br></div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><div dir="ltr">My current very simplistic
prototype, to build a module file, its
respective module object file, and include those
in the library/link for anything that depends on
this library:<br>
<br>
<div><font face="monospace">
add_custom_command(</font></div>
<div><font face="monospace"> COMMAND
${CMAKE_CXX_COMPILER} ${CMAKE_CXX_FLAGS}
-xc++ -c -Xclang -emit-module -fmodules
-fmodule-name=Hello
${CMAKE_CURRENT_SOURCE_DIR}/module.modulemap
-o
${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm
-Xclang -fmodules-codegen</font></div>
<div><font face="monospace"> DEPENDS
module.modulemap hello.h</font></div>
</div>
</blockquote>
<br>
</div>
<div text="#000000" bgcolor="#FFFFFF"> Why does this
command depend on hello.h?</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>Because it builds the binary module interface
(hello_module.pcm) that is a serialized form of the
compiler's internal representation of the contents of
module.modulemap which refers to hello.h (the
modulemap lists the header files that are part of the
module). This is all using Clang's current backwards
semi-compatible "header modules" stuff. In a "real"
modules system, ideally there wouldn't be any
modulemap. Just a .cppm file, and any files it depends
on (discovered through the build system scanning the
module imports, or a compiler-driven .d file style
thing).<br>
<br>
Perhaps it'd be better for me to demonstrate something
closer to the actual modules reality, rather than this
retro header modules stuff that clang supports.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
That would be better for me. I'm interested in modules-ts, but I'm
not interested in clang-modules.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> If that is
changed and module.modulemap is not, what will
happen?</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
If hello.h is changed and module.modulemap is not
changed? The hello_module.pcm does need to be rebuilt.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Hmm, this assumes that the pcm/BMI only contains declarations and
not definitions, right?</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>Nah, it might contain definitions, same as a header file would today (inline) - or non-inline definitions.<br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"> I think clang outputs the definitions in a
separate object file, but GCC currently doesn't. Perhaps that's a
difference that cmake has to account for or pass on to the user.</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>Clang outputs frontend-usable (not object code, but serialized AST usable for compiling other source code) descriptions of the entire module (whatever it contains - declarations, definitions, etc) to the .pcm file. It can then, in a separate step, build an object file from the pcm. I think GCC produces both of these artifacts in one go - but not in the same file.<br></div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_quote"><div>
Ideally all of this would be implicit (maybe with some
flag/configuration, or detected based on new file
extensions for C++ interface definitions) in the
add_library - taking, let's imagine, the .ccm (let's
say, for argument's sake*) file listed in the
add_library's inputs and using it to build a .pcm
(BMI), building that .pcm as an object file along with
all the normal .cc files, <br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Ok, I think this is the separation I described above.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>* alternatively, maybe they'll all just be .cc
files & a build system would be scanning the .cc
files to figure out dependencies & could notice
that one of them is the blessed module interface
definition based on the first line in the file.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Today, users have to contend with errors resulting from their own
code being incorrect, using some 3rd party template incorrectly,
linking not working due to incorrect link dependencies, and
incorrect compiles due to missing include directories (or incorrect
defines specified). I can see incorrect inputs to module generation
being a new category of errors to confuse users.<br>
<br>
For example, if in your list of files there are two files which look
like the blessed module interface based on the first line in the
file, </div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>Two files that both start with "export module foo"? Yes, that would be problematic/erroneous.<br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">there will be something to debug.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>So I suppose the more advanced question: Is there a
way I can extend handling of existing CXX files
(and/or define a new kind of file, say, CXXM?)
specified in a cc_library. If I want to potentially
check if a .cc file is a module, discover its module
dependencies, add new rules about how to build those,
etc. Is that do-able within my cmake project, or would
that require changes to cmake itself? (I'm happy to
poke around at what those changes might look like)<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
One of the things users can do in order to ensure that CMake works
best is to explicitly list the cpp files they want compiled, instead
of relying on globbing as users are prone to want to do: <br>
<br>
<a class="m_2066748816796752644m_-8026624506218320738moz-txt-link-freetext" href="https://stackoverflow.com/questions/1027247/is-it-better-to-specify-source-files-with-glob-or-each-file-individually-in-cmak" target="_blank">https://stackoverflow.com/questions/1027247/is-it-better-to-specify-source-files-with-glob-or-each-file-individually-in-cmak</a><br>
<br>
if using globbing, adding a new file does not cause the buildsystem
to be regenerated, and you won't have a working build until you
explicitly cause cmake to be run again.<br>
<br>
I expect you could get into similar problems with modules - needing
a module to be regenerated because its dependencies change (because
it exports what it imports from a dependency for example). I'm not
sure anything can be done to cause cmake to reliably regenerate the
module in that case. It seems similar to the globbing case to me.<br>
<br>
But aside from that you could probably experimentally come up with a
way to do the check for whether a file is a module and discover its
direct dependencies using file(READ). You might want to delegate to
a script in another language to determine transitive dependencies
and what add_custom{_command,_target} code to generate.</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>OK<br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><br>
<blockquote type="cite">
<div dir="ltr">
<div>But this isn't ideal - I don't /think/ I've
got the dependencies quite right & things
might not be rebuilding at the right times.<br>
Also it involves hardcoding a bunch of things
like the pcm file names, header files, etc.<br>
</div>
</div>
</blockquote>
<br>
</div>
<div text="#000000" bgcolor="#FFFFFF"> Indeed. I think
part of that comes from the way modules have been
designed. The TS has similar issues.</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote"><br>
Sure - but I'd still be curious to understand how I
might go about modifying the build system to handle
this. If there are obvious things I have gotten wrong
about the dependencies, etc, that would cause this not
to rebuild on modifications to any of the source/header
files - I'd love any tips you've got.<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Sure. I didn't notice anything from reading, but I also didn't try
it out. You might need to provide a repo with the
module.modulemap/c++ files etc that are part of your experiment. Or
better, provide something based on modules-ts that I can try out.</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>*nod* I'll see if I can get enough of modules-ts type things working to provide some examples, but there's some more variance/uncertainty there in the compiler support, etc.<br></div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">& if there are good paths
forward for ways to prototype changes to the build
system to handle, say, specifying a switch/setting a
property/turning on a feature that I could implement
that would collect all the .ccm files in an add_library
rule and use them to make a .pcm file - I'd be happy to
try prototyping that.<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
cmGeneratorTarget has a set of methods like GetResxSources which
return a subset of the files provided to add_library/target_sources
by splitting them by 'kind'. You would probably extend
ComputeKindedSources to handle the ccm extension, add a
GetCCMFiles() to cmGeneratorTarget, then use that new GetCCMFiles()
in the makefiles/ninja generator to generate rules.<br>
<br>
When extending ComputeKindedSources could use <br>
<br>
if(Target->getPropertyAsBool("MAKE_CCM_RULES"))<br>
<br>
as a condition to populating the 'kind'. Then rules will only be
created for targets which use something like <br>
<br>
set_property(TARGET myTarget PROPERTY MAKE_CCM_RULES ON)<br>
<br>
in cmake code.<br>
<br>
I'm guessing that's enough for you to implement what you want as an
experiment?</div></blockquote><div><br></div></div></div><div dir="ltr"><div class="gmail_quote"><div>OK, so in that case it requires source changes to cmake? *nod* sounds plausible - I appreciate the pointers. I take it that implies there's not a way I could hook into those file kinds and filters without changing cmake? (ie: from within my project's cmake build files, without modifying a cmake release)</div></div></div><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div>Ideally, at least for a simplistic build, I
wouldn't mind generating a modulemap from all
the .h files (& have those headers listed
in the add_library command - perhaps splitting
public and private headers in some way, only
including the public headers in the module
file, likely). Eventually for the
standards-proposal version, it's expected that
there won't be any modulemap file, but maybe
all headers are included in the module
compilation (just pass the headers directly to
the compiler).<br>
</div>
</div>
</blockquote>
<br>
</div>
<div text="#000000" bgcolor="#FFFFFF"> In a design
based on passing directories instead of files, would
those directories be redundant with the include
directories?<br>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div>I'm not sure I understand the question, but if I
do, I think the answer would be: no, they wouldn't be
redundant. The system will not have precompiled
modules available to use - because binary module
definitions are compiler (& compiler version, and
to some degree, compiler flags (eg: are you building
this for x86 32 bit or 64 bit?)) dependent.</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Right. I discussed modules with Nathan Sidwell meanwhile and
realised this too.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">One of the
problems modules adoption will hit is that all the
compilers are designing fundamentally different
command line interfaces for them. </div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
*nod* We'll be working amongst GCC and Clang at least
to try to converge on something common.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Different flags would not be a problem for cmake at least, but if
Clang didn't have something like -fprebuilt-module-path and GCC did,
that would be the kind of 'fundamental' difference I mean.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div>This also doesn't start to approach the
issue of how to build modules for external
libraries - which I'm happy to
discuss/prototype too, though interested in
working to streamline the inter-library but
intra-project (not inter-project) case first.<br>
</div>
</div>
</blockquote>
<br>
</div>
<div text="#000000" bgcolor="#FFFFFF"> Yes, there are
many aspects to consider.<br>
<br>
Are you interested in design of a CMake abstraction
for this stuff? I have thoughts on that, but I don't
know if your level of interest stretches that far.</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
Not sure how much work it'd be - at the moment my
immediate interest is to show as much
real-world/can-actually-run prototype with cmake as
possible, either with or without changes to cmake
itself (or a combination of minimal cmake changes plus
project-specific recipes of how to write a user's
cmake files to work with this stuff) or also showing
non-working/hypothetical prototypes of what ideal user
cmake files would look like with reasonable/viable
(but not yet implemented) cmake support.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Yes, it's specifying the ideal user cmake files that I mean. Given
that granularity of modules can be anywhere on the spectrum between
one-module-file-per-library and one-module-file-per-class, I think
cmake will need to consider one-module-file-per-library and
*not*-one-module-file-per-library separately. <br>
<br>
In the *not*-one-module-file-per-library case, cmake might have to
delegate more to the user, so it would be more inconvenient for
them.<br></div></blockquote><div><br></div></div></div><div dir="ltr"><div class="gmail_quote"><div>I'm not sure why that would be the case - they'd hopefully be as easy as the other. The same way that header inclusion discovery doesn't require the user to do anything today - though maybe we'd get more thorough checking that you don't depend on modules from libraries you haven't depended upon, that'd be nice.</div></div></div><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<br>
In the one-module-file-per-library case, I think the ideal is
something like:<br>
<br>
add_library(foo foo.cpp)<br>
# assuming foo.h is a module interface file, this creates<br>
# a c++-module called foo and makes it an interface usage <br>
# requirement of the foo target defined above<br>
add_cxx_module(foo foo.h)<br>
<br>
# bar.cpp imports foo.<br>
add_library(bar bar.cpp)<br>
# bar links to foo, and a suitable compile line argument is added
if<br>
# needed for the foo module.<br>
target_link_libraries(bar foo)<br>
<br>
This would work best if foo.h did not contain <br>
<br>
module;<br>
export module foo;<br>
<br>
(after
<a class="m_2066748816796752644m_-8026624506218320738moz-txt-link-freetext" href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0713r1.html" target="_blank">http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0713r1.html</a>)<br>
<br>
but instead contained only <br>
<br>
module;<br>
<br>
and the module name came from the buildsystem (or from the compiler
using the basename).<br>
<br>
As it is, the above cmake code would have to determine the module
name from foo.h and throw an error if it was different from foo.
Having the module name inside the source just adds scope for things
to be wrong. It would be better to specify the module name on the
outside.<br>
<br>
I wonder what you think about that, and whether it can be changed in
the modules ts? My thoughts on this and what the ideal in cmake
would be are changing as the discussion continues.</div></blockquote></div></div><div dir="ltr"><div class="gmail_quote"><div><br>I'm not too involved in the standardization process - though I'm trying to help inform it by getting a better understanding of what's clear, what's not, how it all might come together & helping inform the committee of that. So I'm not sure if/how practical it would be to change that at this stage, but my default assumption would be "not easy" - at least in the sense that, for now, I'm mostly looking at trying to describe how the existing things could work (and what parts of that might be painful - including this name duplication sort of situations like you're pointing out) & likely leaving it to other people on the committee to decide if those pain points are sufficient to consider changing the design.<br> </div></div></div><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">Can you help? It
would really help my understanding of where things
currently stand with modules.</div>
</blockquote>
<div><br>
I can certainly have a go, for sure.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Great, thanks.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> For example, is
there only one way to port the contents of the cpp
files?<br>
</div>
</blockquote>
<div><br>
Much like header grouping - how granular headers are
(how many headers you have for a given library) is up
to the developer to some degree (certain things can't
be split up), similarly with modules - given a set of
C++ definitions, it's not 100% constrained how those
definitions are exposed as modules - the developer has
some freedom over how the declarations of those
entities are grouped into modules.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Yes, exactly. This repo is small, but has a few libraries, so if we
start with one approach we should be easily able to also try a
different approach and examine what the difference is and what it
means.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">After that, is
there one importable module per class or one per
shared library (which I think would make more sense
for Qt)?</div>
</blockquote>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
Apparently (this was a surprise to me - since I'd been
thinking about this based on the Clang header modules
(backwards compatibility stuff, not the
standardized/new language feature modules)) the
thinking is probably somewhere between one-per-class
and one-per-shared-library. But for me, in terms of
how a build file would interact with this, more than
one-per-shared-library is probably the critical
tipping point.</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Yes. I think you're talking about the one-module-file-per-library
and *not*-one-module-file-per-library distinction I mentioned above.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div> If it was just one per shared library, then I'd
feel like the dependency/flag management would be
relatively simple. You have to add a flag to the
linker commandline to link in a library, so you have
to add a flag to the compile step to reference a
module, great. But, no, bit more complicated than that
given the finer granularity that's expected here.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
"finer granularity that's *allowed* here" really. If there is a
simple thing for the user to do (ie one-module-file-per-library),
then cmake can make that simple to achieve (because the dependencies
between modules are the same as dependencies between targets, which
the user already specifies with target_link_libraries). <br>
<br>
If the user wants to do the more complicated thing
(*not*-one-module-file-per-library), then cmake can provide APIs for
the user to do that (perhaps by requiring the user to explicitly
specify the dependencies between modules).<br>
<br>
My point is that cmake can design optimize for the easy way and I
think users will choose the easy way most of the time.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div> </div>
</div>
</div>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">The git repo is
an attempt to make the discussion concrete because
it would show how multiple classes and multiple
libraries with dependencies could interact in a
modules world. I'm interested in what it would look
like ported to modules-ts, because as far as I know,
clang-modules and module maps would not need porting
of the cpp files at all.</div>
</blockquote>
<div><br>
Right, clang header-modules is a backwards
compatibility feature. It does require a constrained
subset of C++ to be used to be effective (ie:
basically your headers need to be what we think of as
ideal/canonical headers - reincludable, independent,
complete, etc). So if you've got good/isolated
headers, you can port them to Clang's header modules
by adding the module maps & potentially not doing
anything else - though, if you rely on not changing
your build system, then that presents some problems if
you want to scale (more cores) or distribute your
build. Because the build system doesn't know about
these dependencies - so if you have, say, two .cc
files that both include foo.h then bar.h - well, the
build system runs two compiles, both compiles try to
implicitly build the foo.h module - one blocks waiting
for the other to complete, then they continue and
block again waiting for bar.h module to be built. If
the build system knew about these dependencies (what
Google uses - what we call "explicit (header)modules")
then it could build the foo.h module and the bar.h
module in parallel, then build the two .cc files in
parallel.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
I think that the 'build a module' step should be a precondition to
the compile step. I think the compiler should issue an error if it
encounters an import for a module it doesn't find a file for. No one
expects a linker to compile foo.cpp into foo.o and link it just
because it encounters a fooFunc without a definition which was
declared in foo.h. <br>
<br>
That would reduce the magic and expect something like <br>
<br>
add_cxx_module(somename somefile.h otherfiles.h)<br>
<br>
to specify a module file and its constituent partitions, which I
think is fine.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div>Basically: What do folks think about
supporting these sort of features in CMake C++
Builds? Any pointers on how I might best
implement this with or without changes to
CMake?</div>
</div>
</blockquote>
<br>
</div>
<div text="#000000" bgcolor="#FFFFFF"> I think some
design is needed up front. I expect CMake would want
to have a first-class (on equal footing with include
directories or compile definitions and with
particular handling) concept for modules, extending
the install(TARGET) command to install module binary
files etc. <br>
</div>
</blockquote>
<div><br>
</div>
<div>Module binary files wouldn't be installed in the
sense of being part of the shipped package of a
library - because module binary files are
compiler/flag/etc specific.</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div><div text="#000000" bgcolor="#FFFFFF">
Ok.<br>
<br>
Thanks,<br>
<br>
Stephen.<br>
<br>
</div>
</blockquote></div></div></div></div>