[cmake-developers] Current deficiencies of automoc

Alan W. Irwin irwin at beluga.phys.uvic.ca
Sat Oct 22 16:10:17 EDT 2016


On 2016-10-22 07:44+0100 Sascha Cunz wrote:

>> Compilers know exactly where the header files are through
>> include_directories or the equivalent target PROPERTY so why can't
>> automoc find those same header files using similar logic?
>
> In short: Because you really do not want automoc to do that.
>
> A bit longer; highlighting the semantic problem:
>
> The following applies to targets that you define as well as the external ones. The external one that you certainly can’t skip is QtCore, so I’ll explain it in terms of Qt5::Core.
>
> By adding Qt5::Core to your target’s dependencies at least one -I option is added to the compiler, so it knows where to find Qt5 Core’s header files. At the same time you might have a subdirectory in your source tree that contains some of your header files that you also want to be searched, so you add i.e. “target_include_directories(MyTgt ${PROJECT_SOURCE_DIR}/FooModule)”. Now, the list of include directories for MyTgt contains two entries: Qt5::Core’s headers and FooModule. They are not distinguishable by any means. The question now is: How should automoc know which files you want to run through moc and which files not?
>
> There are actually 3+1 scenarios now:
>
> 1. qcoreapplication.h - If you run automoc on that file, your target ends up defining the meta data for QCoreApplication, which is inefficient and will either lead to undefined behaviour or on most platforms to linker errors.
> Independent of the technical things below, the decision to run or not run such headers through moc seems to be impossible to automate - One would need to either white or black list the directories used / not used for automoc.
>
> 2. FooModule/file1.h, which is included from file1.cpp in MyTgt - You might want to run that through moc, but actually only in the case that the compiler is building MyTgt. If the compiler is building another one of your targets that happens to require this include path, you end up the same as in case 1 above.
>
> 3. FooModule/file2.h, which does not have a corresponding .cpp file in MyTgt - (if this was on the system level: for example, not all files in /usr/include belong to the same project) This will lead to unresolved symbols in MyTgt (on most platforms) and the inability to actually execute MyTgt, unless it is linked against the library that actually contains the .cpp file (on all platforms).
>
> Then there is even more obscure cases:
> 4. Imagine the files x/x.cpp and x/public/x.hpp (Both belonging to target X). Now, someone decides that the target Y (with code in y/) doesn’t need all the code in target X but just that one class. So, they add the file x/x.cpp to target X and target Y. If x/x.cpp contains the line “#include “public/x.hpp”, then target Y doesn’t even need the -I for x/public. How should automoc figure out that the file needs to be run through moc anyway?
>
> And even longer with a bit of historic and technical background and focus on the technical problem:
>
> The automoc concept and original implementation had been developed for the KDE project. IIRC, the original purpose was to speed up compilation by reducing the absolute number of translation units (i.e. number of .cpp files == number of compiler runs == number of linker inputs). This was achieved by two mechanism:
>
> A) All header files that contained Q_OBJECT/GADGET are collected and run through moc, but the results of that were not added as translation units but rather were #include’ed in _one_ translation unit, which then was added to the target.
>
> b) All source files that contained Q_OBJECT/GADGET needed to include the “<name>.moc” file - if such an include was spotted, the .cpp file itself was run through moc.
>
> This tool was implemented outside of cmake and integrated into the build system as an additional target, on which the consuming target had a dependency. Much later, it was integrated into cmake, keeping this concept for various reasons.
>
> Now back to the cmake implementation of automoc: When automoc is enabled for target T, cmake will add an additional target to the build - let’s call it TAM. T has a dependency on TAM, so TAM is always build _before_ T even starts to scan for its dependencies (in makefile based builds). Note that some generators don’t even have the requirement to scan for dependencies at compile time (i.e. MSBuild / Visual Studio does that by itself).
> Scanning for dependencies is basically: “Run the preprocessor and tell me what files it would read if it were to do the actual work, then create a file with rules to describe these dependencies”.
>
> Because when the cmake generator is run, it can’t actually know if there are indeed files that need to run through moc, it unconditionally adds the moc_T.cpp file to T (This is a problem for my current customer, because they have 500 targets where automoc is enabled based on whether Qt5::Core is a dependency or not - but only about 100 of these targets need automoc in the first place and it happens that Qt5::Core is a very low level dependency for them). The moc outputs that are #include’ed don’t need to be added to the target.
> The problem here is, that once a target is created in CMake, you can’t add any files to it. So this has to be done that way.
>
> Since the TAM target is build before T’s “scan for dependencies”, automoc can’t know what files _exactly_ are included in the project (unless given through add_executable/add_library). This makes an informed automatic decision in the above cases 3 and 4 practically impossible. And for case 1, this means: We’d actually have to consider _all_ files in _all_ -I directories, possibly including /usr/include and /usr/local/include.
>
> So, what we’re left with is the actual command lines to the compiler, the include paths as defined in CMakeLists.txts. This _could_ be used to solve the case 2. But such a solution would be very error prone and still leaves case 1 to be sorted out somehow.
>

Hi Sascha:

I accept what you have said, and I very much appreciate your attempt
to educate me (and others here) in these C++/Qt/moc technical
considerations.  Therefore, subject to some minor corrections such as
the subsequent one by Craig Scott I strongly urge you to put this
essential background information into the CMake FAQ so it is not lost
in the large traffic on this list.


> I’ve read the cmake-qt(7) section about automoc several times now. I can’t actually figure out anything inside it that is plain _wrong_. However, I would suggest to change the documentation to:
>
> 	The AUTOMOC target property controls whether cmake(1) inspects the C++ header and implementation
> 	files in the target to determine if they require moc to be run, and to create rules to execute moc at the
> 	appropriate time.
>
> 	Note that it is recommended to explicitly list header files when declaring a target with add_executable
> 	or add_library - even though this is only required when header and corresponding implementation file
> 	are in different directories. {1}

That statement is not true in complete generality so I would suggest
the addition of the following qualifier

"Note that it is recommended" ==> 
"Note that to assure consideration for moc processing it is recommended"

Also, it sounds from your footnote 1 that you would prefer not to have
all the details here about the exact rules for identifying the header
files to be scanned so I would suggest the following addition to the
above paragraph.

When the header is in the same directory as the C++ implementation
file there are additional possibilities for identifying it for
consideration for moc processing, see the automoc documentation.

And then the automoc documentation should be updated with those
same-directory possibilities clearly stated.

>
> 	moc needs to be run, if:
> 	- A Q_OBJECT or Q_GADGET macro is found in a header file. In this case, the result will be put into
> 	into a file named according to moc_<basename>.cpp. Multiple moc outputs generated that way will
> 	be added to the target as one translation unit.
>
> 	- The macro is found in a C++ implementation file. Following the Qt conventions, moc output will be
> 	put into a file named according to <basename>.moc.
> 	The implementation file may optionally use a preprocessor #include to include the .moc file (This is
> 	typically done at the end of the file). If such an include is omitted, the .moc file will also be added to
> 	the aforementioned translation unit.
>
> 	Generated moc_*.cpp and *.moc files are placed in the build directory so it is convenient to set the
> 	CMAKE_INCLUDE_CURRENT_DIR variable. {2}
>
> 	The moc command line will consume the COMPILE_DEFINITIONS and INCLUDE_DIRECTORIES
> 	target properties from the target it is being invoked for, and for the appropriate build configuration.
>
> 	The AUTOMOC target property may be pre-set for all following targets by setting the
> 	CMAKE_AUTOMOC variable. The AUTOMOC_MOC_OPTIONS target property may be populated
> 	to set options to pass to moc. The CMAKE_AUTOMOC_MOC_OPTIONS variable may be populated
> 	to pre-set the options for all following targets.

> {1} I’d consider this too generic for _that_ part of the
> documentation, but it could be added: "Header files that are in the same
> directory as their implementation file will be automatically added to
> the target”.
> {2} Maybe this should actually be changed to “so it might be required
> to set”

Assuming you have no strong objections to the above further changes,
then I plan to wrap up our joint cmake-qt documentation changes into git
format-patch form for the convenience of the CMake developers here.
And I also plan to work on the automoc documentation using similar
language to what you have used above in the cmake-qt case with some
additions about the same-directory possibilities for identifying
headers that should moc'ed.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________


More information about the cmake-developers mailing list