[cmake-developers] Some numbers on cmake+ninja+VS on Windows

[Peter Kümmel]

On 18.02.2012 21:27, Óscar Fuentes wrote:
> Setup: Windows XP SP3 32 bits running on VMWare Workstation 8.0.1 with 4
> real cores (no HT) assigned out of 4 available on the host machine,
> which is Kubuntu 11.08 64 bits (kernel version 3.0.0-16-generic).
>
> Visual Studio 2010 Ultimate. JOM 0.9.4 (that's an old version, but on my
> experience newer versions are slower).
>
> CMake was taken from https://github.com/syntheticpp/CMake.git, branch
> ninja-generator-pr-win revision fa9ce5e, plus this patch:
>
> diff --git a/Modules/Platform/Windows-cl.cmake b/Modules/Platform/Windows-cl.cmake
> index ccccbc9..be6abb6 100644
> --- a/Modules/Platform/Windows-cl.cmake
> +++ b/Modules/Platform/Windows-cl.cmake
> @@ -37,7 +37,7 @@ SET(CMAKE_COMPILE_RESOURCE "rc<FLAGS>  /fo<OBJECT>  <SOURCE>")
>   # that is automatically copied into try_compile directories
>   # by the global generator.
>   SET(MSVC_IDE 1)
> -IF(CMAKE_GENERATOR MATCHES "Makefiles")
> +IF(CMAKE_GENERATOR MATCHES "Makefiles" OR CMAKE_GENERATOR MATCHES "Ninja")
>     SET(MSVC_IDE 0)
>     IF(NOT CMAKE_VC_COMPILER_TESTS_RUN)
>       SET(CMAKE_VC_COMPILER_TESTS 1)
> @@ -125,7 +125,7 @@ IF(CMAKE_GENERATOR MATCHES "Makefiles")
>       ENDIF(CMAKE_COMPILER_RETURN)
>       MAKE_DIRECTORY("${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/CMakeTmp3")
>     ENDIF(NOT CMAKE_VC_COMPILER_TESTS_RUN)
> -ENDIF(CMAKE_GENERATOR MATCHES "Makefiles")
> +ENDIF(CMAKE_GENERATOR MATCHES "Makefiles" OR CMAKE_GENERATOR MATCHES "Ninja")
>
>   IF(MSVC_C_ARCHITECTURE_ID MATCHES 64)
>     SET(CMAKE_CL_64 1)
> diff --git a/Source/cmNinjaTargetGenerator.cxx b/Source/cmNinjaTargetGenerator.cxx
> index e48ac12..034aefe 100644
> --- a/Source/cmNinjaTargetGenerator.cxx
> +++ b/Source/cmNinjaTargetGenerator.cxx
> @@ -230,9 +230,13 @@ cmNinjaDeps cmNinjaTargetGenerator::ComputeLinkDeps() const
>     if(!cli)
>       return cmNinjaDeps();
>
> -  const std::vector<std::string>  &deps = cli->GetDepends();
> -  cmNinjaDeps result(deps.size());
> -  std::transform(deps.begin(), deps.end(), result.begin(), MapToNinjaPath());
> +  cmNinjaDeps result;
> +  for(unsigned i = 0; i<  cli->GetItems().size(); ++i) {
> +    if( cli->GetItems()[i].Target ) {
> +      result.push_back( cli->GetItems()[i].Target->GetName() );
> +    }
> +  }
> +
>     return result;
>   }
>
>
> Ninja was taken from https://github.com/syntheticpp/ninja.git, branch
> token-splitter revision 335e08e.
>
> I worked with some internal projects and with LLVM revision f340a29
> taken from LLVM's official git mirror: http://llvm.org/git/llvm.git
>
> First, some quick recap of the build tool I used so far with VS:
>
> NMake: no support for parallel jobs.
>
> MSBuild (equivalent to VS IDE): platform checks very slow, tricky
> support for parallel jobs (one knob /m:JOBS at MSBuild.exe level and
> another indepent one /MP at the compiler driver level; it's easy to end
> with starved cores or with dozens of compile jobs competing for the
> available cores.) On addition, CMake+MSBuild suffers from a nasty bug
> that consists on not being able to use the regenerated project files
> when CMake is invoked from within the build because some CMakeLists.txt
> changed. There more issues with MSBuild when it is used as a
> replacement of `make' but those described so far are enough to motivate
> anyone to search for a better alternative.
>
> Jom: no support for building more than one top-level target in
> parallel. This was reported to the author time ago as a feature request
> and was closed as wontfix.
>
> Now, for the numbers.
>
> Build LLVM with -DCMAKE_BUILD_TYPE=Release:
>
> ninja -j 4: 6m55s
> MSBuild (cmake --build .): 38m17s (*)
> jom: 10m45s
>
> (*): this very long time may be due to the lack of top-level
> parallelization (no /m:JOBS was used for invoking msbuild.exe) and/or a
> slower console when executing on VMWare for showing all the text output
> that msbuild emits.
>
> No-op build (best of two consecutive runs):
>
> ninja -j 4: 0.4s
> MSBuild: 11s
> jom: 53s
>
> `ninja -t clean' takes less than a second.
>
> On an internal project here, a full build with ninja requires half the
> time of jom/msbuild (35s vs 1m20s) while a no-op build is instantaneous
> with ninja (0.1s) while reasonably fast with msbuild (2.3s) and
> annoyingly slow with jom (8.5s). That project consists on a shared
> library with ~30 cpp files plus 10 small (typically 1 cpp file) shared
> libraries or executables that depends on the big shared library. On this
> scenario, Ninja's correct handling of parallel jobs makes a lot of a
> difference over jom and msbuild.
>
> CMake + Ninja looks like a perfect match for people building with VC++
> outside of the IDE.
>

I've uploaded a cmake installer for Windows which contains above patches
and also ships a ninja binary:

     http://sourceforge.net/projects/cmakescript/files/cmake-2.8.7.20120202-Ninja.exe

When installing be sure there are NO spaces in the installation patch!

Ninja automatically uses all available cores. Therefore sometimes the build stops
because a file is already locked by another process, then call ninja with -j1.

Feel the speed ;)

Peter