[cmake-developers] CMake master slowdown in generation step

[Stephen Kelly]

Brad King wrote:

> On 04/04/2014 11:13 AM, Stephen Kelly wrote:
>> The Ninja generator was much slower than the Makefiles generator. I
>> pushed the optimize-source-file-processing topic with a commit which
>> should fix the major problem with Ninja. I'll see if there is more
>> opportunity for optimizing/caching though in general.
> 
> I see this commit in that topic:
> 
>  Ninja: Query custom commands once per target, not once per file.
>  http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=6c202d15
> 
> Nils, your example just had a lot of source files but no custom
> commands.  Is the slowdown observable with Makefile generators?

The existence of custom commands doesn't matter so much. The difference is 
calling cmGeneratorTarget::GetCustomCommands 2000 times vs calling it once. 
It involves cmMakefiles::GetOrCreateSource and creating a lot of 
cmSourceFileLocations etc:

 http://www.steveire.com/callgrind.out.Makefiles
 http://www.steveire.com/callgrind.out.Ninja

(test case reduced to 200 files).

I've pushed another commit which gets the time for the 2000 file testcase 
from about 20s to about 4s for both Ninja and Makefiles. It fails a few unit 
tests though and I don't have time right now to find out why. 4s is still a 
bit much, so there's probably more opportunity.

Thanks,

Steve.