[CMake] Obtaining improved GNU make performance on Makefiles generated by cmake

[Alan W. Irwin]

On 2008-03-24 10:33-0400 Bill Hoffman wrote:

> OK, I talked with Brad, and found out the difference from 2.4.8 and CVS. 
> When make runs in a CMake project, the first thing it does is run cmake to 
> check the depend information.  It looks for missing header files, and other 
> checks to see if CMake should re-run or not.  In 2.4.8, CMake would do this 
> globally for the whole project.  These made a delay as CMake checked all the 
> depends for the whole project.  CMake now checks each targets header files 
> separately.   This means two things. First, if you are only building one 
> target, then it should be much faster as all the headers for the entire 
> project are not checked. Second, the initial delay for the project is much 
> less, but over all time may increase some as the check is now done for each 
> target and shared headers maybe double checked.

That sounds like a good working hypothesis for the substantially increased
latency of the cvs version, but you should probably confirm that analysis
with some -d runs to make absolutely sure. If that hypothesis is confirmed,
and if your judgement is most developers simply type "make" the majority of
the time (which is what I tend to do) without bothering with individual
targets, then you may want to reconsider that change because of the
substantially increased latency penalty associated with it.

>
> That said, I have checked in a change to get PHONY targets into CMake, as it 
> should not break other makes.  This should help some, and it should fix the 
> make test problem when you had a directory called test.

"make test" sparked this investigation, but I have long since solved that
issue by renaming our test subdirectory as plplot_test since we are a
cross-platform product where some of our Unix users do not have access to
GNU make.

So the emphasis for most of this thread has simply focussed on the important
issue of reducing the latency for GNU make users, and I really appreciate
your attempts to address that issue.

Unfortunately, the results for your newly implemented .PHONY support for GNU
make are disappointing for some unknown reason.

Just to review, the PLplot make latency for cmake 2.4.8 is

software at raven> time make >& /dev/null
real    0m1.454s
user    0m0.800s
sys     0m0.724s

and for cmake version 2.7-20080320 is is

real    0m2.319s
user    0m1.356s
sys     0m1.068s

with timing errors of +/- 0.01 s or so for each measurement.

For cmake version 2.7-20080324 (which includes your .PHONY changes as I
checked with actual results in the build-tree Makefiles) that latency has
been increased by a small amount that appears to be above the timing errors.

software at raven> time make >& /dev/null

real    0m2.375s
user    0m1.444s
sys     0m1.028s

I notice cmake_force is still in the generated Makefiles for cmake
2.7-20080324. It's count is reduced compared to the number of .PHONY
instances.

software at raven> find -type f |xargs grep cmake_force |wc -l
824
software at raven> find -type f |xargs grep .PHONY |wc -l
5849

Both the above commands were executed from the top of the build tree.

You should be able to eliminate cmake_force altogether for GNU make since
.PHONY duplicates its function.  My guess from the count above is that
will make a noticable difference.

I think it requires a CMake and make guru to find out what is really going
on using a through investigation comparing cmake 2.4.8 with cmake
2.7-20080320 and cmake 2.7-20080324 with the make -d option.  I am not that
guru (all I know is that make -d outputs an enormous number of results which
are hard for me to understand), but if one of the CMake developers wants to
investigate further, PLplot should be a reasonably accessible test bed for
such investigations.  Note with PLplot there are a large number of options
to simplify the build.  For example, you easily can eliminate all the non-C
language interfaces and/or the devices to simplify comparisons.  You should
probably check other projects as well including a simple "hello-world"
project if timing errors don't completely swamp the results in that case.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________