[CMake] Generating dependencies with gcc -M

[Michael Hertling]

On 06/29/2010 08:40 AM, Tom Birch wrote:
> 
> On Jun 28, 2010, at 10:09 AM, Michael Hertling wrote:
> 
>> On 06/28/2010 05:24 AM, Tom Birch wrote:
>>> CMake's dependency scanner uses its own parser to scan for #include directives, and then builds up the dependency tree this way. I know it's possible to rig up an invocation of gcc -M to generate the correct dependencies, and then feed this into the OBJECT_DEPENDS property of source files, but that means that dependency generation would happen when running 'cmake .', not 'make'.
>>
>> One compelling reason why dependency scanning is delayed until building
>> time is that it's taking dynamically generated files into account, i.e.
>> files not being present at configuration time. Look at the following
>> CMakeLists.txt:
>>
>> CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
>> PROJECT(DYNDEP C)
>> FILE(WRITE ${CMAKE_BINARY_DIR}/f.h "void f(void);\n")
>> FILE(WRITE ${CMAKE_BINARY_DIR}/f.c "\#include \"f.h\"\nvoid f(){}\n")
>> FILE(WRITE ${CMAKE_BINARY_DIR}/main.c.in
>>    "\#include \"f.h\"\nint main(void){f(); return 0;}\n"
>> )
>> ADD_CUSTOM_COMMAND(
>>    OUTPUT main.c
>>    COMMAND cp main.c.in main.c
>>    DEPENDS ${CMAKE_BINARY_DIR}/main.c.in
>> )
>> INCLUDE_DIRECTORIES(".")
>> ADD_EXECUTABLE(main ${CMAKE_BINARY_DIR}/main.c ${CMAKE_BINARY_DIR}/f.c)
>>
>> After cmaking, when running make, the dependency of main.c.o on f.h is
>> figured out, and this couldn't be achieved during the configuration as
>> main.c doesn't exist at that time.
> 
> Heh, you are always the one to answer my questions!

Well, I try to do my best. ;)

> In this case, something in the makefiles has to generate the dependencies for this generated .c file, and today that is cmake's dependency scanner. I'm arguing that this could be replaced by gcc -M or the platform-specific equivalent. Most makefile-based buildsystems do this today.

What is the platform-specific equivalent? Another one of CMake's
strengths is the freedom in choosing the compiler, so you may decide,
e.g., not to use gcc even in a typical Linux environment. Perhaps, I'm
cross-compiling for a special target using a vendor-specific toolchain,
and in that situation, I'd dislike to see gcc performing the dependency
scanning. Moreover, it's possible that gcc or any external scanner isn't
installed at all, so you must be prepared to handle this case, e.g. by a
user's choice. In other words, replacing the CMake dependency scanner by
"gcc -M" or the like means, IMO, complicating things and increasing the
whole build process's interrelation with external tools, and as for me,
I'm in doubt if this is worth it. Furthermore, look at notes 15546 and
15552 to <http://public.kitware.com/Bug/view.php?id=8561> to see that
CMake's dependency scanner addresses another issue.

>>> I guess the bigger question here is: why doesn't cmake use gcc -M internally when it's available? It's vastly superior to any homegrown parser, so why not use it?
>>
>> My assumption is: As gcc or other tools for dependency scanning like
>> makedepend are not available or desired on all systems supported by
>> CMake there's a need for an in-house solution, at least as fallback,
>> and if you once have to provide such a solution why not using it
>> thoroughly? Besides, this reduces the dependencies on external
>> programs - one of CMake's strengths.
> 
> That is true, but CMake has to know how to invoke various different compilers, and generate various different project files (XCode, VC++, etc). Since gcc (and every gcc-compatible compiler) supports -M and VC++ supports /showIncludes, why can't CMake present an abstraction to the user that does correct dependency discovery? Is there really another mainstream compiler out there which doesn't support this kind of dependency generation, that outweighs the benefits of generating dependency lists in a robust, foolproof manner?

While CMake usually does a good job in setting up the toolchains to use
one can read about related difficulties on this list every now and then,
so adding a dependency scanner to compiler, linker et al. as a further,
say, external functionality would probably tend to escalate problems of
this kind. Keeping the number of necessary programs to a minimum is a
good idea, I think, and dependency scanning - in contrast to compiling
and linking - is a task that can be unhesitatingly done by CMake itself.

Nevertheless, w.r.t. adding new languages it would be interesting if one
could hook into CMake's built-in dependency scanning. Currently, AFAIK,
CMake doesn't support it for languages other than the premier ones like
C and C++, so I wonder if one could resort to an external scanning tool
for a new language, e.g. via a rule variable CMAKE_<LANG>_SCAN_DEPENDS.
Perhaps, a CMake developer can tell us if there are considerations to
provide an open interface for dependency scanning in this or another
manner.

BTW, if you want to use "gcc -M", e.g. to populate the OBJECT_DEPENDS
property, you may do so: EXECUTE_PROCESS() to run "gcc -M", collecting
the output in an OUTPUT_VARIABLE and tweak it as needed with STRING().
Possibly, you can even do such a custom scanning operation at build
time: <http://www.mail-archive.com/cmake@cmake.org/msg26212.html>

Regards,

Michael