[cmake-developers] Generating buildsystem metadata from CMake

Anton Makeev Anton.Makeev at jetbrains.com
Thu Apr 2 04:29:21 EDT 2015


> On 01 Apr 2015, at 23:13, Stephen Kelly <steveire at gmail.com> wrote:
> 
> Anton Makeev wrote:
> 
>>> On 01 Apr 2015, at 00:41, Stephen Kelly
>>> <steveire at gmail.com> wrote:
>>> 
>>> Anton Makeev wrote:
>>> 
>>>> The problem with the following format:
>>>> ---
>>>> "include_directories" : ["/foo", "/opt"]
>>>> "compile_definitions" : ["DEF=\"Foo\"", "OTHER_DEF=1"]
>>>> "compile_flags": [ "-c" ]
>>>>>>>> Is that it’s incomplete and cannot be used directly:
>>>> * include directories list misses compiler-defined search paths
>>> 
>>> CMake gathers those during configure time, and we can add them to the
>>> metadata too.
>> 
>> However appealing it is, I don’t think it’s a good option,
> 
> Ok, I understand now, because the used built-in defines are different more 
> often than the built-in include directories (-iquote and -I- are rarely used 
> in my experience, but -std=, -ansi, -fPIC which all cause defines to be 
> added, are common).
> 
>> We’d rather do it lazily when times come to process the given
>> file having specialized caches on IDE side.
> 
> Ok. Do you currently use compile-commands.json for this?

At the moment we don’t use it; it's mostly because at the time of development we weren’t aware of
this file, and, then, we got all the necessary information from existing files, generated by CMake.

We were considering to try using this file exactly when this discussion was started; so I’m not sure if we are 
going to use it or wait for this proposal to make it’s way in CMake.

>>> Are you asking for that as an option? That would be an independent
>>> feature request. I'd suggest filing it in the tracker.
>>> Otherwise, the only way CMake currently would pass such options to the
>>> compiler is if the user specified them in CMAKE_CXX_FLAGS or
>>> add_compile_options() or similar.
>> 
>> Yes, I mean that user may easily have -iquote directly passed as command
>> line option and the IDE need to know that.
> 
>> (e.g. include_directories may some day have a flag to indicate whether it
>> “” or <>)
> 
> Ok, I filed 
> 
> http://public.kitware.com/Bug/view.php?id=15491
> 
> Please add any additional information you have, or any design ideas for how 
> the command interface should be.

Thanks, Stephen

>> The current proposal (with having separate include_directories,
>> compile_definitions, compile_flags) will force the IDE to assemble the
>> compiler-specific command line itself that is quite prone to errors and
>> will require constant maintenance since additional options may appear
>> (e.g. include_directories may some day have a flag to indicate whether it
>> “” or <>)
> 
> I agree. However, there are two separate use-cases to address.
> 
> 1) Get information from CMake so that the IDE can understand the C++
> 2) Get information to understand the CMake code.
> 
> You are talking about the first, but I think the second is useful too and an 
> IDE could offer features related only to the CMake code (ie unrelated to the 
> c++ code) based on that information (such as 'Add include directory' 
> action).

I see, but how will having include_directories help ‘Add include directory’-like actions?
I mean, simply having this list (without additional info like backtraces) doesn’t seem to help much.

> So, in the end, we probably need tooling to be able to get the information 
> in both forms. Ie, as an array of directories, and as a string of compiler 
> command line options.
> 
> We already have the latter in the form of compile-commands.json. One of the 
> reasons I wonder if you are using that already is to know whether 
> 
> 1) That is already a good enough solution 
> or 
> 2) The new metadata file should contain similar information
> 
> I'm wondering if we can learn anything from your experience with the 
> existing file.

I’m afraid we don’t have much to contribute here, since we don’t use this file at the moment.
After a brief investigation, it seems to have the necessary command lines we expect in the form
that can be easily parsed (apart from  -o <path> -c <path> that will need to be cut out, but it’s not a big deal).

Just to make sure, you mentioned that this file should be deprecated with this new metadata.json,
is that correct?

>> Given the details above, please let me know, if you still need them,
>> I’ll try to collect the specific cases that we support from our test
>> suite.
> 
> I think the details you provided above are enough for understanding, but 
> getting some cases from your test suite relating to this stuff would indeed 
> be interesting, I expect.

OK, I’ll post the examples in a separate message in a couple of days.


>> We currently reconstruct the command line from the
>> '<build_dir>/CMakeFiles/Target1.dir/flags.make’ file for each target,
> 
> I see. There are several disadvantages to that, among them being that it 
> ties you to makefiles (until you duplicate your work to support Ninja), and 
> the structure of those files should not be considered 'stable'. CMake should 
> be able to change the way it generates the buildsystem-specific files in any 
> release.

Fully agree and I hope we’ll come up with a good stable solution.


>> Here is possible options to get this info from metadata.json as I see it:
>> 
>> 1) every file has a full associated '<compiler_path> <target_defines>
>> <file_defines> <target_flags> <file_flags>' command line. This will
>> produce lot of duplication, though; but it’s the best option for IDE
>> integration, since we don’t need any additional logic in the IDE.
> 
> It also seems similar to (but not exactly the same as) the existing optional 
> JSON file compile-commands.json.
> 
> What do you think of the option of generating both something like
> 
> "targets": {
>  "foo": {
>    "include_directories": ["/opt/foo", "/opt/bar"],
>    "quote_include_directories": ["/opt/quote1", "/opt/quote2"],
>    "all_compile_flags": "-I/opt/foo -I/opt/bar -iquote /opt/quote1 ...",
>    "sources": [
>      "file1.cpp",
>      "file2.cpp",
>      { 
>        "file3.cpp": 
>        {
>          "all_compile_flags":
>                "-I/opt/foo (etc, as above) -DFILE3_ONLY -Wall"
>        } 
>      },
>      "file4.cpp",
>    ]
>  }
> }
> 
> for a CMakeList.txt file like
> 
> add_executable(foo file1.cpp file2.cpp file3.cpp file4.cpp)
> target_include_directories(foo PRIVATE "/opt/foo" "/opt/bar")
> target_include_directories(foo QUOTE PRIVATE "/opt/quote1" "/opt/quote2")
> set_source_files_properties(file3.cpp PROPERTIES 
>   COMPILE_FLAGS "-DFILE3_ONLY -Wall"
> )
> 
> 
> That is, the "all_compile_flags" in the target JSON-object contains the same 
> as what CMake actually generates (need to check what it does for IDEs). 
> Those flags apply to all sources specified in the target object, except for 
> those source files which specify exceptions. Exceptions are specified by a 
> JSON-object instead of a string in the "sources" array. If it's an object 
> with a "all_compile_flags" property, then that overrides the property of the 
> same name on the target JSON-object, and again, it is what CMake actually 
> generates for the source file in the buildsystem.
> 
> Does that make sense? That means you would read the "all_compile_flags" for 
> the specific source file if it is present, and use the "all_compile_flags" 
> for the target otherwise (which you already have cached).

That sounds like a brilliant idea. This way we’ll have minimum duplication and minimum logic required on
the IDE-side.

> I know the "include_directories" array is redundant/duplicated information, 
> but as it is provided for a different purpose, I think that is ok.

Agree, It won’t hurt for sure.


Regards,
Anton Makeev
JetBrains
http://www.jetbrains.com
"Develop with pleasure!"

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2856 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20150402/a0d7ec4e/attachment.bin>


More information about the cmake-developers mailing list