[cmake-developers] Setting includes, defines and other usage requirements with one command

[Stephen Kelly]

Brad King wrote:
> Meanwhile, I've noticed the test run times on some dashboard machines
> have doubled since two weeks ago.  Can you please run some timing
> tests comparing current 'next' to 2.8.10.2 for some existing projects?

Actually they seem to have doubled overnight when tll-includes-defines was 
merged:

 http://open.cdash.org/viewTest.php?buildid=2743648
 http://open.cdash.org/viewTest.php?buildid=2795799

I pushed a patch which is needed for using the new interfaces in boost, and 
which speeds up the cmake time in kde frameworks branch a bit:

 before tll-includes-defines
 real    0m27.528s
 user    0m16.721s
 sys     0m6.732s

 after tll-includes-defines
 real    0m30.767s
 user    0m19.965s
 sys     0m6.624s

 After optimize:
 real    0m27.246s
 user    0m16.661s
 sys     0m6.388s


Before the optimization, callgrind shows that a lot of time is spent in 
GetIncludeDirectories, in particular in executing 
cmSystemTools::ExpandListArgument with the result of genex evaluation.

Also, after optimization and with kde frameworks updated to populate and 
rely on target interfaces, I get this:

 real    0m26.922s
 user    0m16.401s
 sys     0m6.404s

It will be interesting to see if the optimization has any affect on the 
dashboards. The effect of GetIncludeDirectories taking a long time is 
multiplied several times there for the multiconfig generators. However, the 
optimization should only have any effect at all if several targets are used 
which have dependencies between them, and where there are multiple paths to 
the same dependencies.

I considered also doing some caching in GetIncludeDirectories  similar to 
the caching done in the linking related methods, but it didn't seem to have 
a noticable difference. At least for the makefile generator, 
GetIncludeDirectories is called only twice with the same arguments. 

The optimization only works because we know that includes and defines are 
uniq'd after the generator expressions are evaluated. 


Looking into the future a bit, I think that as new transitive interfaces are 
added in the future (COMPILE_OPTIONS and LINK_OPTIONS), it might make sense 
to keep that invariant with those too. Otherwise the performance problem 
would likely re-appear.

In the case of COMPILE_OPTIONS at least, order can be relevant in some 
cases. 

 http://gcc.gnu.org/onlinedocs/gcc-3.4.4/gcc/Optimize-Options.html

> If you use multiple -O options, with or without level numbers, the last 
> such option is the one that is effective. 

That would mean that if we have something like this:

 target_compile_options(foo INTERFACE -O3)
 target_compile_options(bar INTERFACE -O2)
 target_link_libraries(foo PRIVATE bar)

 target_link_libraries(user PRIVATE bar foo)

then the un-optimized result would look like:

 -O2 -O3 -O2

because foo has INTERFACE_COMPILE_OPTIONS "-O3;$<LINKED:bar>" and user would 
end up with COMPILE_OPTIONS "$<LINKED:bar>;$<LINKED:foo>" which expands 
first to "-O2;-O3;$<LINKED:bar>" and then "-O2;-O3;-O2".

However the 'optimized' result would look like:

 -O2 -O3

because the first expansion would be "-O2;-O3;$<LINKED:bar>", but the 
repeated 'bar' would be ignored.

If instead of 

 target_link_libraries(user PRIVATE bar foo)
 
it was 

 target_link_libraries(user PRIVATE foo bar)

then the optimized result would be:

 -O3 -O2

and the unoptimized result would be 

 -O3 -O2 -O2 


The example of -O flags is not a particularly good one as it's not likely 
something suitable for putting into an INTERFACE. However, other compile 
options which would be suitable for the INTERFACE in theory might have 
unexpected results with this kind of thing.

Solutions I can think of include declaring that it is not supported to rely 
on ordering in COMPILE_OPTIONS, and adding a $<NO_SKIP:...> expression to 
temporarily turn off the optimization. 

For example, if we had this:

 target_link_libraries(foo PRIVATE bar)
 target_compile_options(foo INTERFACE -O3)
 target_compile_options(bar INTERFACE -O2)

 target_link_libraries(user PRIVATE foo bar)

(note the order changed)

the optimized result would be "-O2;-O3" even though bar is listed last in 
'user', because that is skipped. This could then be used instead where 
needed:

 target_link_libraries(user PRIVATE foo $<NO_SKIP:bar>)

Alternatively, in the case of COMPILE_OPTIONS, we could make 'top-level' 
items always NO_SKIP.

Just some food for thought for when we get around to this stuff.

Thanks,

Steve.