[CMake] Visual Studio <Version> <Arch> - Ninja Generator

[James Johnston]

Hi,

The problem with /MP in cl.exe, /CGTHREADS in link.exe, and /MAXCPUCOUNT in msbuild.exe is they don't communicate with each other, potentially leading to massive over/undersubscription of the system.  I would love it if Microsoft could get this mishmash cleaned up somehow so they can all communicate and not oversubscribe the system.  (Out-of-the-box, Visual C++ ought to have this straightened out).  In theory it would be faster than Ninja because the /MP switch reduces forking overhead and fixed overhead initialization cl.exe by batch building.

In the meantime, Ninja seems faster in practice, because it can properly avoid oversubscription with building individual C++ files and entire projects in parallel (whereas cl.exe and msbuild.exe cannot). 

(One problem with link.exe /CGTHREADS though is that Ninja needs to be told how many threads link.exe will actually use, so that Ninja doesn't oversubscribe.  I'm not aware of a way to make Ninja do that though.  You can use pools to restrict to one link at a time, but moderate oversubscription will still happen with the concurrent cl.exe instances.)

Best regards,

James Johnston

From: CMake [mailto:cmake-bounces at cmake.org] On Behalf Of Hendrik Sattler
Sent: Monday, August 31, 2015 15:49
To: Guillaume Dumont; cmake at cmake.org
Subject: Re: [CMake] Visual Studio <Version> <Arch> - Ninja Generator

Hi,

did you really enable parallel compiling in VS with /MP24 or the like? Note that using devenv to build uses the number from the IDE user settings but using msbuild needs a command line option.

Else just create a custom target that calls cmake for ninja and afterwards ninja itself in yet another build tree. you can limit the creation of that target to if(MSVC_IDE).

HS

Am 31. August 2015 16:35:01 MESZ, schrieb Guillaume Dumont <dumont.guillaume at gmail.com>:
Hi all, 
I would like to know what kind of effort would it take to generate a Visual Studio generator that bypasses the normal MSBuild build and uses Ninja instead. I have been working on different projects which build much faster with ninja than MSBuild (several orders of magnitude on a machine with 24 cores). Still I like to use the Visual Studio environment to use the debugger. Using ninja is especially useful for projects with a lot of CUDA files which are built sequentially for every target using MSVS.
I would like to contribute but I don't really know where to start and if such an effort already exists.