[cmake-developers] Generating buildsystem metadata from CMake

[Anton Makeev]

> On 01 Apr 2015, at 00:41, Stephen Kelly <steveire at gmail.com> wrote:
> 
> Anton Makeev wrote:
> 
>> The problem with the following format:
>> ---
>> "include_directories" : ["/foo", "/opt"]
>> "compile_definitions" : ["DEF=\"Foo\"", "OTHER_DEF=1"]
>> "compile_flags": [ "-c" ]
>> —
>> Is that it’s incomplete and cannot be used directly:
>> * include directories list misses compiler-defined search paths
> 
> CMake gathers those during configure time, and we can add them to the 
> metadata too.

However appealing it is, I don’t think it’s a good option, since it'll make generation
phase way longer, since files and targets may have different command line options and
CMake will need to call compiler for all of them. We’d rather do it lazily when times come
to process the given file having specialized caches on IDE side.


>> * quote include directories are not distinguishable
> 
> CMake does not currently generate -iquote as a result of a user writing an 
> include_directories() call. 
> 
> Are you asking for that as an option? That would be an independent feature 
> request. I'd suggest filing it in the tracker.
> Otherwise, the only way CMake currently would pass such options to the 
> compiler is if the user specified them in CMAKE_CXX_FLAGS or 
> add_compile_options() or similar. 

Yes, I mean that user may easily have -iquote directly passed as command line
option and the IDE need to know that.

> CMake does not attempt to parse CMAKE_CXX_FLAGS for things which look like 
> include directories. That variable is just an opaque string of data from the 
> point of view of CMake. It is not clear to me whether you are suggesting 
> that CMake should do that. If you do want to suggest that, please file an 
> issue in the tracker.
> Similarly, CMake does not attempt to analyse the meaning of options 
> specified in add_compile_options(). Again, it's not clear to me whether you 
> are suggesting a new feature for CMake. If you are, please file it in the 
> tracker.

No, I don’t suggest CMake should do that, sorry for the confusion.
I was just explaining why we need the whole compiler-specific command line, instead of 
a list of separate flags and include directories  - we’ll pass that command line 
as is (actually with some post-processing) to the compiler to collect the actual
defines, header search paths etc. (some examples below)

The current proposal (with having separate include_directories, 
compile_definitions, compile_flags) will force the IDE to assemble the compiler-specific 
command line itself that is quite prone to errors and will require constant maintenance
since additional options may appear (e.g. include_directories may some day have a
flag to indicate whether it “” or <>)

> To avoid misunderstandings, please post snippets of CMake code together with 
> a description of what information you want CMake to determine from the 
> snippet and what metadata it should generate.

Not sure what you are asking, I hope I made it more clear with previous explanation.

>> * compiler definition do not include compiler-defined(built-in)
>> definitions etc.
> 
> CMake does not currently gather those during configure, as far as I know. 
> That could be added though.

It'd rather not, since the set may be different for the specific files.

>> That is, to have the complete list of include directories and compiler
>> definitions, the IDE will have to call the compiler anyways. And here is
>> where it will need the actual compiler’s command line.
>> 
>> Should we had the full list of include directories (split into groups
>> like: <>, quote, and frameworks) and the list of all compiler defines, we
>> would not need the compiler command-line at all.
> 
> I think I understand what you're saying. However, it would help if you 
> related what you write to existing CMake features and snippets of CMake 
> code. I assume that you are medium-to-very familiar with CMake features 
> already, given what you are working on?

Given the details above, please let me know, if you still need them,
I’ll try to collect the specific cases that we support from our test suite.

>> Though I don’t this it’s a
>> way to go for CMake generation, since the compiler should be called for
>> every unique command-line flags and undoubtedly make generation way
>> longer. 
> 
> It's not completely clear to me what you are suggesting here. Maybe you know 
> something I don't know.
> 
> Can you be clear about what you are suggesting CMake would have to do for 
> each unique command line? I know you are not proposing that CMake should do 
> it (you said the IDE should do it lazily instead), but what action are you 
> actually talking about? What should the IDE do lazily instead of CMake doing 
> it? Invoke the compiler to determine the compiler-defined/built-in include 
> directories and definitions? They will always be the same, right? Or are you 
> talking about the "-I-" GCC option here and how that clears the built-ins? 
> Please be more specific.

We currently reconstruct the command line from the '<build_dir>/CMakeFiles/Target1.dir/flags.make’ file
for each target, that lists per-target flags and defines, and per-files flags and defines if any:
—
# compile C with /usr/bin/cc
C_FLAGS = -isysroot macosx10.8 -I/.../include_path -common-c-flag   -target1-flag "-flag with spaces"
C_DEFINES = -DTarget2DefA=1 -DTarget2DefB=1

# Custom flags: CMakeFiles/Target1.dir/Foo.o_FLAGS = -foo-file-flag
# Custom defines: CMakeFiles/Target1.dir/Foo.o_DEFINES = FooDef=1
—

This info is enough to construct the complete compiler command line using the pattern
<compiler_path> <target_defines> <file_defines> <target_flags> <file_flags>:

For the above example it will be:
/usr/bin/cc -DDefWithSpaces="value \"with\" spaces" -DTarget2DefA=1 -DTarget2DefB=1
    -DTarget2DefC=1 -DFooDef=1  -isysroot macosx10.8 -I/…/include_path -common-c-flag  -target1-flag "-flag with spaces” -foo-file-flag

We then add additional flags like -dD -M etc. to get the necessary information from the compiler
and call (compiler is actually called only once for the unique set of options with cashing etc.).
And we have the most accurate defines and include paths for the code model, that we can possibly get (afaik).


Here is possible options to get this info from metadata.json as I see it:

1) every file has a full associated '<compiler_path> <target_defines> <file_defines> <target_flags> <file_flags>' command line.
This will produce lot of duplication, though; but it’s the best option for IDE integration, 
since we don’t need any additional logic in the IDE.

2) as you have originally proposed, metadata could have separate properties like
lang_flags, lang_defines, 
target_flags, target_defines, 
file_flags, file_defines,
plus a rule how to merge these flags together for the compiler (in what order).
In this case, we’ll have no duplication in the metadata file and minimum logic required
on IDE side to build the command line

3) the info in the form of
"include_directories" : ["/foo", "/opt"]
"compile_definitions" : ["DEF=\"Foo\"", "OTHER_DEF=1"]
"compile_flags": [ "-c” ]

This is a worst scenario for the IDE, since not only will it need to reconstruct the command line,
it will also have to know, how to pass these include_directories and compile_definitions to the specific 
compiler  is prepend each directory with -I, and each definition with -D for gcc and clang).

I hope I haven’t confuse you even more with this explanation.


Regards,
Anton Makeev
JetBrains
http://www.jetbrains.com
"Develop with pleasure!"

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2856 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20150401/ed7f7c95/attachment.bin>