[cmake-developers] Generating buildsystem metadata from CMake

[Aleix Pol]

On Sat, Mar 14, 2015 at 10:02 AM, Stephen Kelly <steveire at gmail.com> wrote:
>
>
> On Saturday, March 14, 2015, Aleix Pol <aleixpol at kde.org> wrote:
>> On Wed, Mar 11, 2015 at 11:10 AM, Stephen Kelly <steveire at gmail.com>
>> wrote:
>>> Hi,
>>>
>>> Following from the thread here:
>>>
>>>
>>> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12394
>>>
>>> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12650
>>>
>>> I'm starting to gather requirements and make sure the feature is
>>> well designed to satisfy the needs we're already aware of, and fits with
>>> the features CMake currently has.
>>>
>>> The aim is to generate a structured file containing metadata relating the
>>> buildsystem.
>>>
>>> To help with completing the design of this feature, I've written
>>> documentation (documentation driven design), and a unit test
>>> containing a CMakeLists.txt file which exercises many modern CMake
>>> features in the "generate-metadata" branch in my clone.
>>>
>>> Mostly the design I propose can be read in the documentation I wrote. I'm
>>> interested in any feedback.
>>>
>>>
>>> https://gitorious.org/cmake/steveires-cmake/source/generate-metadata:Tests/Metadata/CMakeLists.txt
>>>
>>> http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.html
>>>
>>> I expect to require a few iterations to figure out what the metadata
>>> files
>>> should contain in the end.  Note that there are already some differences
>>> between my design and Aleix's implementation, such as that my design
>>> proposes one metadata file per config. There are also some things
>>> missing like location, because it is not yet clear to me whether build
>>> or install locations are needed etc.
>>>
>>> The content of the metadata file is determined by the build properties,
>>> and
>>> is necessarily similar to the compile-related content created when
>>> generating the actual buildsystem.  It additionally contains information
>>> about the output locations of build artifacts and information relating to
>>> the cmake description itself.
>>>
>>> Goals include:
>>>
>>> * Make it possible for IDEs to access the compile-related information for
>>>   autocompletion and code navigation etc purposes.
>>>
>>> * Remove the need for IDEs to parse generated Makefiles or Ninja files to
>>>   access compile-related information.  The structure of those files is
>>> not
>>>   'stable', while the content of the metadata file is stable.
>>>     http://thread.gmane.org/gmane.comp.programming.tools.cmake.user/48412
>>>
>>>
>>> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=11081
>>>
>>> * Remove the need for users to create a new build directory and new build
>>>   in order to use or switch IDEs.  QtCreator requires that
>>>   the C::B 'extra generator is used as it parses compile information from
>>>   that.  Other 'extra generators' such as for eclipse, sublime, kate etc
>>>   also require fresh/new build directories, although the actual
>>> buildsystem
>>>   they create is identical (assuming using all Makefile based or
>>>   all Ninja based 'extra generators')
>>>
>>> * Make it possible to write a plugin for the editors/IDEs such as sublime
>>>   which consumes the metadata file and invokes the build using whatever
>>>   buildsystem the user already has a build directory for, instead of
>>>   writing an 'extra generator' and maintaining it in the cmake repo.
>>>     http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/9004
>>>
>>> * Make it possible for editors/IDEs to allow specifying the configuration
>>>   at build-time, where the IDE has that feature, and where a multi-config
>>>   generator is available.  That is, QtCreator provides user-interface for
>>>   choosing debug/release config to build.  Currently it can't offer that
>>>   when using cmake, because it only allows the use of Makefile or Ninja
>>>   generators, in order to make use of the C::B file.  QtCreator should be
>>>   able to use the Xcode or Visual Studio generators, generate the
>>> metadata
>>>   file(s), and invoke `cmake --build . --config ${CONFIG}` as
>>>   appropriate.  Eclipse, Sublime and other editors have similar abilities
>>>   to invoke config-specific builds after generation.
>>>
>>> * Provide a list of 'build targets', which can be listed and invoked in
>>>   IDE/editor user interface.  Build targets for all linked binaries
>>>   and utilties are provided.  The tooling is expected to perform
>>> filtering
>>>   on the target types to show only executables and utilities for
>>>   execution, for example.
>>>
>>> * Provide a list of source files per target per type of source file, eg
>>>   object sources, header files, generated files, files excluded from the
>>>   active configuration/platform/compiler, non-compiled files.
>>>
>>> * Make it more easy for an IDE to support actions such as 'remove file
>>>   from the project', which requires removing it from the CMakeLists.txt
>>>   at the appropriate place, and 'add new file/class to target', which
>>>   involves adding code to the CMakeLists.txt file at the appropriate
>>>   place.  Most likely the easiest way to do the latter is using the
>>>   target_sources() command, and to support the former, the location of
>>>   the declaration of the target, and all target_sources() calls would
>>>   need to be recorded.  Even that is not enough because of transitive
>>>   consumption of source files through the link interface, but that is
>>>   likely irrelevant.
>>>
>>> * Provide information about the entire build graph of link-dependencies
>>>   for visulization and dependency analysis
>>>     http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/4042
>>>
>>> * Provide enough information about runtime link dependencies for IDEs to
>>>   be able to properly invoke targets with debuggers, profilers and other
>>>   tools.
>>>
>>>
>>> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=11011
>>>
>>>
>>> The CMAKE_EXPORT_COMPILE_COMMANDS variable already exists to generate
>>> compile command lines, but that is a file whose format is defined by
>>> Clang, not by CMake, and other tools in the Clang ecosystem also support
>>> it.  It is not extensible or versioned in the way that a CMake-defined
>>> file format is, so I don't propose using it as a starting point.
>>>
>>> Notes:
>>>
>>> * I deliberately didn't push an implementation to my repo, so that this
>>>   discussion can focus on the design of the feature and contents of the
>>>   file first.
>>>
>>> * In the documentation and the test I didn't mention that the metadata
>>> file
>>>   is JSON.  I'm not opposed to it, but I think there should be some kind
>>> of
>>>   schema defining how to read the metadata file.  If we go with JSON
>>> (instead of xml, say), then we'd have to define our own schema format,
>>> which
>>>   is not impossible.  My proposal is to generate the schema beside the
>>> metadata
>>>   file instead of writing a metadata version into the metadata file
>>> itself.  Does
>>>   anyone feel strongly about the file being JSON?  I just want to record
>>> the
>>>   reason for everything in this design phase - we don't have to spend a
>>> lot
>>>   of time on it.
>> [1]
>>
>>>
>>> * I didn't document the location or directory.  I'm not clear on whether
>>>   it is supposed to be the build location, or the install location(s!),
>>>   or all of those.
>>>
>>> * I don't generate 'dependencies' (actually the list of files which the
>>>   buildsystem re-generation depends on) as Aleix did, because there is no
>>>   well-defined usefulness for that list yet.
>> [2]
>>
>>>
>>> * I documented the 'name' for the targets, and the TARGET_FILE_NAME
>>>   etc information, and can push an implementation which generates them.
>>>
>>>
>>> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12619
>>>
>>> * If it is useful to preprocess/compile/assemble individual files from
>>>   IDEs, as made possible by the Makefiles and Ninja generators, we'll
>>> need
>>>   to design that.
>>>
>>>
>>> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12429
>>>
>>> * Some more information from project() may be relevant, but it's not
>>> clear
>>>   yet. We will likely know more when we have decided the file format and
>>>   generated some 'interesting' metadata files.
>>>
>>> * We might need backtraces for not only all add_library and
>>> add_executable
>>>   calls, but also all target_sources() calls.
>>>
>>> * We list the Generator used to configure the build.  That way IDEs know
>>>   which build output parser to use.
>>>
>>> * I propose to generate one metadata file per configuration active in the
>>>   buildsystem.  That means that the Makefile and Ninja generators would
>>>   generate only one file, and the multi-config generators would generate
>>>   multiple files.  When changing the 'active configuration', IDEs would
>>>   then read whatever metadata file is relevant to the active
>>> configuration.
>>>
>>>   This also means that conditions don't need to be added inside the
>>> metadata
>>>   file for configurations in order to show files 'excluded' by being part
>>> of
>>>   a non-active configuration.  The code implementing discovery of
>>> excluded
>>>   files is in the generate-metadata branch in my clone.
>>>
>>>
>>>
>>> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=11323
>>>     "At least Qt Creator does not need information on which conditions to
>>>      be met for a file to become part of the current build."
>> [3]
>>
>>>
>>> * I propose generating language-specific source lists, because CMake can
>>>   build files with compilers which do not match their file extension.
>>>
>>> * I propose generating language-specific compile properties in the
>>> metadata
>>>   file if there a language specific variant groups.  For example if the
>>>   compile options themselves are language-specific (the feature enabling
>>> that
>>>   was merged to CMake master this week), then a property for each
>>> language
>>>   is specified.  Otherwise only a language-agnostic property is
>>> generated. This
>>>   can vary per-configuration for a multi-config generator, so each
>>> metadata
>>>   file for a multi-config generator could have different properties
>>> present.
>>>   Consumers read the language specific property if it is present, and
>>> read
>>>   the language agnostic version otherwise.
>>>
>>> * Generating metadata only (without generating buildsystem files) is not
>>>   currently in scope.  This was requested several times, but it is not
>>>   clear why.
>> [4]
>>
>>>
>>> * How much information does tooling need about installation?  Targets
>>>   can use different include directories and compile definitions in their
>>>   install locations compared to their build locations.  If IDEs want to
>>>   provide some user interface related to the project files in their
>>>   install location, perhaps a separate solution based on cmExportFile*
>>>   is needed.  For future investigation.
>> [5]
>>
>> Hi,
>> To be honest, I don't really understand why we're doing this here now,
>> 7 months after the first e-mail with the proposal of the feature. I'm
>> hoping we're doing this for transparency and, to be honest, all I want
>> by now is this to be over.
>>
>> [1] About schema, you mean this, no? http://json-schema.org/examples.html
>> I think it's fine to come up with a schema, can be useful to provide
>> tests I guess?
>
> That looks great after a short glance, yes.
>
> It should be useful not only for unit tests, but for telling tools how to
> parse the metadata files when multiple different versions of the content can
> be generated by cmake.
>
>> I don't think it's important.
>> I think JSON is the format to use, but then anything that's easily
>> parseable works for me. JSON is, much more than XML even.
>
> OK, I'm happy with using json too.
>
>> [2] Ok. The KDevelop implementation was using it for being able to
>> know what files are in scope.
>
> By 'in scope' you mean 'used by the primary CMakeLists.txt files via
> include() or find_package()' or something like that? OK, let's see if that's
> the best source of that data or if we should add something new internally in
> cmake.
By in scope I mean whether a variable or a target defined in such files.

>
>
>> I'm still planning to do some cmake
>> parsing for code-completion and it seemed like a way to provide the
>> information. We can consider this part of the second iteration of the
>> project though, as it wasn't very useful as it was anyway.
>>
>> [3] This is the biggest problem I see with this proposal. This has 2
>> problems:
>> * We will have to set up a file system watcher to keep track of new
>> cmake-metadata*.ext files popping up at random times.
>
> The 'ext' was just a placeholder while I wasn't sure about the data format
> to generate. Let's call it cmake-metadata.json now.
>
> The Makefile and Ninja generators will only ever generate and keep
> up-to-date a single 'cmake-metadata.json' without any -debug or -release
> suffix, because those are 'single configuration generators'. Xcode and
> Visual Studio will generate the configuration specific files, but will
> always regenerate all of them.
Ok.

>
>> * When the configs change, the file will remain there as cmake doesn't
>> clean up stuff generated from previous builds. This will confuse the
>> IDE since it will have available files that are not available anymore.
>
> I think the above explanation covers this.
>
>> [4] Because cmake is really slow and you often want to have the
>> metadata file updated often, every time one of the cmake files in the
>> project change.
>
> OK, maybe we can add something for that, but is it the generation stage that
> is the slow part? How much time would we gain by doing this?
Well, I'm unsure what's the best way to go about it. I understand that
given the json file will be executed after all the processing of the
sources it won't help much if we just skip the generation. Also maybe
it's possible to optimize the configure process all together.

>
>>
>> [5] In fact, installation information is more interesting for headers
>> than targets, IMHO.
>
> Can you say why?
To be able to use the headers that are being developed instead of the
installed ones when developing 2 projects that are under development.
Let's leave that for a later iteration.

Aleix