[CMake] [cmake-developers] Obtaining header file dependencies of a source file manually

[iosif neitzke]

What does output_required_files() [0] do, and is it applicable here?

[0] https://cmake.org/cmake/help/v3.4/command/output_required_files.html

On Mon, Nov 30, 2015 at 2:09 AM, Petr Kmoch <petr.kmoch at gmail.com> wrote:
> Hi Dan,
>
> you could look into the IMPLICIT_DEPENDS argument of add_custom_command:
> https://cmake.org/cmake/help/latest/command/add_custom_command.html
>
> I don't have direct experience with it, but it looks like it could do what
> you're looking for.
>
> Petr
>
> On Sun, Nov 29, 2015 at 10:47 AM, Dan Liew <dan at su-root.co.uk> wrote:
>>
>> Hi,
>>
>> # TL;DR
>>
>> I need a way of determining the header file dependencies of a source
>> file and inform CMake about them. CMake doesn't do this automatically
>> because I'm using custom commands for the compilation step so CMake
>> doesn't do it's usual magic of automatically inferring source file
>> header dependencies. This is because this part of the compilation step
>> requires a separate C++ compiler (completely independent from the one
>> that CMake detects at configure time).
>>
>> Is there a way of doing this that will also regenerate the
>> dependencies if the source file changes?
>>
>> # Long version
>>
>> A C++ project that I work on [1] is (amongst other things) a compiler.
>> In order to compile applications it needs to link in a supporting
>> runtime that is compiled to LLVM bitcode which is linked into the
>> applications it compiles.
>>
>> The supporting runtime is written in C++ and compiled with Clang. The
>> compilation of the runtime is currently achieved using
>> ``add_custom_command()`` and so I am not using CMake's in built
>> support for detecting header file dependencies. The reason for doing
>> it this way is because the LLVM bitcode compiler (i.e. Clang) **is
>> not** the same compiler as the host C++ compiler for the project. For
>> example the host code might be built with MSVC but the supporting
>> runtime is **always** built with Clang.
>>
>> To see this concretely take a look at [2].
>>
>> The build works correctly if you build from a clean build directory.
>> It does not work correctly if you perform a rebuild and change one of
>> the header files that the supporting runtime depends on. This is
>> because CMake doesn't know that the runtime source files depend on the
>> header files and so doesn't rebuild the relevant source files.
>>
>> I can obviously tell CMake about these manually (adding more entries
>> under ``DEPENDS`` in the ``add_custom_command()`` invocation) but this
>> is very cumbersome.
>>
>> What I really need is a way to
>>
>> * Automatically infer the dependencies of a source file and tell CMake
>> about them
>> * Have the dependencies automatically regenerated if the source file
>> changes (someone could add or remove a header file include).
>>
>> In a simple Makefile build system this typically achieved by doing
>> something like this:
>>
>> ```
>> all:: $(SRCS:.cpp:.o)
>>
>> SRCS := foo.cpp bar.cpp
>>
>> # Include SRC file dependencies generated by the compiler if they exist
>> -include $(SRCs:.cpp=.d)
>>
>> %.o : %.cpp
>>   $(CXX) -c $< -o $@ -MP -MMD -MF $*.d
>> ```
>>
>> Note the ``-M*`` flags get the compiler when it runs to generate
>> additional makefile rules that will get included next time a build
>> happens.
>>
>> I don't really know how to do the same thing with CMake. One idea is
>> at configure time invoke Clang with the ``-MP -MMD -MF`` flags on each
>> runtime source file, extract the dependencies then pass them to
>> ``DEPENDS`` in ``add_custom_command()``. If I wanted the dependencies
>> regenerated if the runtime source file changes then I would need to
>> somehow get CMake to reconfigure every time this happens.
>>
>> I don't like this idea very much due to
>>
>> * Having to invoke Clang manually to determine the dependencies. CMake
>> already knows how to determine source file dependencies, but this
>> functionality (AFAIK) isn't exposed to me.
>>
>> * Reconfiguring every time one of the runtime source file changes is
>> annoying (configuring can be slow sometimes).
>>
>> Does anyone have any other ideas? CMake obviously knows how to do all
>> this stuff already for source files being compiled for the detected
>> host C++ compiler, I just don't know how to get at this logic for
>> source files that need to be built with a second independent C++
>> compiler.
>>
>> [1] https://github.com/halide/Halide
>> [2] https://github.com/halide/Halide/blob/master/src/CMakeLists.txt#L140
>>
>> Thanks,
>> Dan.
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake-developers
>
>
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake