[cmake-developers] Current deficiencies of automoc

[Sascha Cunz]

> Compilers know exactly where the header files are through
> include_directories or the equivalent target PROPERTY so why can't
> automoc find those same header files using similar logic?

In short: Because you really do not want automoc to do that.

A bit longer; highlighting the semantic problem:

The following applies to targets that you define as well as the external ones. The external one that you certainly can’t skip is QtCore, so I’ll explain it in terms of Qt5::Core.

By adding Qt5::Core to your target’s dependencies at least one -I option is added to the compiler, so it knows where to find Qt5 Core’s header files. At the same time you might have a subdirectory in your source tree that contains some of your header files that you also want to be searched, so you add i.e. “target_include_directories(MyTgt ${PROJECT_SOURCE_DIR}/FooModule)”. Now, the list of include directories for MyTgt contains two entries: Qt5::Core’s headers and FooModule. They are not distinguishable by any means. The question now is: How should automoc know which files you want to run through moc and which files not?

There are actually 3+1 scenarios now:

1. qcoreapplication.h - If you run automoc on that file, your target ends up defining the meta data for QCoreApplication, which is inefficient and will either lead to undefined behaviour or on most platforms to linker errors.
Independent of the technical things below, the decision to run or not run such headers through moc seems to be impossible to automate - One would need to either white or black list the directories used / not used for automoc.

2. FooModule/file1.h, which is included from file1.cpp in MyTgt - You might want to run that through moc, but actually only in the case that the compiler is building MyTgt. If the compiler is building another one of your targets that happens to require this include path, you end up the same as in case 1 above.

3. FooModule/file2.h, which does not have a corresponding .cpp file in MyTgt - (if this was on the system level: for example, not all files in /usr/include belong to the same project) This will lead to unresolved symbols in MyTgt (on most platforms) and the inability to actually execute MyTgt, unless it is linked against the library that actually contains the .cpp file (on all platforms).

Then there is even more obscure cases:
4. Imagine the files x/x.cpp and x/public/x.hpp (Both belonging to target X). Now, someone decides that the target Y (with code in y/) doesn’t need all the code in target X but just that one class. So, they add the file x/x.cpp to target X and target Y. If x/x.cpp contains the line “#include “public/x.hpp”, then target Y doesn’t even need the -I for x/public. How should automoc figure out that the file needs to be run through moc anyway?

And even longer with a bit of historic and technical background and focus on the technical problem:

The automoc concept and original implementation had been developed for the KDE project. IIRC, the original purpose was to speed up compilation by reducing the absolute number of translation units (i.e. number of .cpp files == number of compiler runs == number of linker inputs). This was achieved by two mechanism:

A) All header files that contained Q_OBJECT/GADGET are collected and run through moc, but the results of that were not added as translation units but rather were #include’ed in _one_ translation unit, which then was added to the target.

b) All source files that contained Q_OBJECT/GADGET needed to include the “<name>.moc” file - if such an include was spotted, the .cpp file itself was run through moc.

This tool was implemented outside of cmake and integrated into the build system as an additional target, on which the consuming target had a dependency. Much later, it was integrated into cmake, keeping this concept for various reasons.

Now back to the cmake implementation of automoc: When automoc is enabled for target T, cmake will add an additional target to the build - let’s call it TAM. T has a dependency on TAM, so TAM is always build _before_ T even starts to scan for its dependencies (in makefile based builds). Note that some generators don’t even have the requirement to scan for dependencies at compile time (i.e. MSBuild / Visual Studio does that by itself).
Scanning for dependencies is basically: “Run the preprocessor and tell me what files it would read if it were to do the actual work, then create a file with rules to describe these dependencies”.

Because when the cmake generator is run, it can’t actually know if there are indeed files that need to run through moc, it unconditionally adds the moc_T.cpp file to T (This is a problem for my current customer, because they have 500 targets where automoc is enabled based on whether Qt5::Core is a dependency or not - but only about 100 of these targets need automoc in the first place and it happens that Qt5::Core is a very low level dependency for them). The moc outputs that are #include’ed don’t need to be added to the target.
The problem here is, that once a target is created in CMake, you can’t add any files to it. So this has to be done that way.

Since the TAM target is build before T’s “scan for dependencies”, automoc can’t know what files _exactly_ are included in the project (unless given through add_executable/add_library). This makes an informed automatic decision in the above cases 3 and 4 practically impossible. And for case 1, this means: We’d actually have to consider _all_ files in _all_ -I directories, possibly including /usr/include and /usr/local/include.

So, what we’re left with is the actual command lines to the compiler, the include paths as defined in CMakeLists.txts. This _could_ be used to solve the case 2. But such a solution would be very error prone and still leaves case 1 to be sorted out somehow.

> 
> I have already suggested some possible changes to the
> documentation on the initial thread on the CMake mailing list and this
> additional thread here on the CMake developer list.
> But if nobody is
> inspired to take immediate action on fixing the problem summarized
> above (which would largely make the current documentation correct), I will write
> up these documentation issues for the current automoc behaviour for
> the bugtracker as well.

With the above background added to your knowledge, you might want to rethink if you consider this a bug in the automoc implementation. 

I’ve read the cmake-qt(7) section about automoc several times now. I can’t actually figure out anything inside it that is plain _wrong_. However, I would suggest to change the documentation to:

	The AUTOMOC target property controls whether cmake(1) inspects the C++ header and implementation
	files in the target to determine if they require moc to be run, and to create rules to execute moc at the
	appropriate time.

	Note that it is recommended to explicitly list header files when declaring a target with add_executable
	or add_library - even though this is only required when header and corresponding implementation file
	are in different directories. {1}

	moc needs to be run, if:
	- A Q_OBJECT or Q_GADGET macro is found in a header file. In this case, the result will be put into
	into a file named according to moc_<basename>.cpp. Multiple moc outputs generated that way will
	be added to the target as one translation unit.

	- The macro is found in a C++ implementation file. Following the Qt conventions, moc output will be
	put into a file named according to <basename>.moc.
	The implementation file may optionally use a preprocessor #include to include the .moc file (This is
	typically done at the end of the file). If such an include is omitted, the .moc file will also be added to
	the aforementioned translation unit.

	Generated moc_*.cpp and *.moc files are placed in the build directory so it is convenient to set the
	CMAKE_INCLUDE_CURRENT_DIR variable. {2}
	
	The moc command line will consume the COMPILE_DEFINITIONS and INCLUDE_DIRECTORIES
	target properties from the target it is being invoked for, and for the appropriate build configuration.

	The AUTOMOC target property may be pre-set for all following targets by setting the
	CMAKE_AUTOMOC variable. The AUTOMOC_MOC_OPTIONS target property may be populated
	to set options to pass to moc. The CMAKE_AUTOMOC_MOC_OPTIONS variable may be populated
	to pre-set the options for all following targets.

Cheers
Sascha

{1} I’d consider this too generic for _that_ part of the documentation, but it could be added: "Header files that are in the same directory as their implementation file will be automatically added to the target”.
{2} Maybe this should actually be changed to “so it might be required to set”