[cmake-developers] NMake Batch-Mode Rules (was: Inquiry on contribution to NMake generator)

[Nagy-Egri MĂĄtĂŠ Ferenc]

Thanks Brad for the detailed description. From what I’ve read about the 3 level makefile system, it should be pretty straightforward to implement batch-mode support.


Actual compilation takes place only in the last level, and throughout generation, this is the point where all the compilation options are present. Obtaining a list of files with identical compiler switches might be tricky. Depends on how CMake handles this in code internally. I’ll see once I dive into code. Deferred generation of the makefile is not nice, but might be the only way. Handling the exact location of the object files might be another property source files must match.


As far as I’ve seen NMake is not able to call multiple makefiles in parallel, so source-level parallelism might be the most NMake could do, but for large projects that could be sufficient.






Feladó: Brad King
Elküldve: ‎szerda‎, ‎2015‎. ‎május‎ ‎27‎. ‎20‎:‎13
Címzett: Nagy-Egri MC!tC) Ferenc
Másolat: cmake-developers at cmake.org





On 05/27/2015 01:33 PM, Nagy-Egri MC!tC) Ferenc via cmake-developers wrote:
> like to see the NMake generator finally support multicore builds by
> adding Batch Mode support to the generated makefiles.

For reference, Batch-Mode Rules are documented here:

 https://msdn.microsoft.com/en-us/library/f2x0zs74.aspx

They support specifying multiple source files to a single "cl" invocation,
as in

 cl -c src1.cpp src2.cpp

instead of

 cl -c src1.cpp
 cl -c src2.cpp

Any multi-core utilization occurs inside "cl" and NMake is not aware of it.

> I have read on the CMake user mailing list, that the problem with
> Batch Mode support, is that it is not trivial to implement.

It is very hard to implement and may not be possible at all.  See below.

>   * What is the design of the NMake generator in CMake?
>   * What is the pourpose of having so many makefiles?

http://www.cmake.org/Wiki/CMake_FAQ#Why_does_CMake_generate_recursive_Makefiles.3F

>   * What is the reason why it is not trivial to implement batch mode?

The main reasons are:

* CMake generates a separate compilation line for each source in each
  target and the flags may differ across sources and targets.  CMake
  does not use pattern rules for any make backend.

* CMake needs to control the location of the object files and "cl" does
  not allow this when more than one source is given.

Implementing batch mode rules would require teaching the generator to
somehow recognize when a group of sources are all built with the same
flags and generate a batch mode rule that matches exactly those sources
and no others (likely the hardest part).  This grouping would also have
to account for the names that "cl" generates for the object files to avoid
conflicts within a group.  I do not know if generating a proper batch mode
rule is possible in general.

>   * Where should I start browsing the source files?

One place is Source/cmLocalUnixMakefileGenerator3.cxx but there is no
specific spot to make a change like this.  It would likely involve
major refactoring.

One can achieve multi-core builds without third-party tools by using
the VS IDE generators.  It is still possible to build from the command
line with these.

-Brad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20150601/4a8d2d93/attachment.html>