[CMake] Deferred CPU count variable in CMake for Makefile generator?

[David Cole]

On Fri, Sep 9, 2011 at 10:49 AM, Clifford Yapp <cliffyapp at gmail.com> wrote:
> Per the customizing Makefile email, I'd like to ask if a mechanism exists or
> could be implemented to allow CMakeLists.txt files to define commands that
> pass a CPU count to subcommands - e.g. something that allows the generators
> to take a syntax along the lines of:
>
> ADD_CUSTOM_TARGET(subbuild
>      COMMAND ${CMAKE_COMMAND} --build ${TARGET_DIR} -- -j${PARENT_CPUS}
> )
>
> and generate makefiles that will take "make JOBS=3 subbuild" and have cmake
> in turn launch "make -j3" in the target directory - in other words,
> ${PARENT_CPUS} would be a cue for the backend generator to add logic to pass
> down parallel make settings to the subbuild command.  (Ideally -j3 could be
> passed down, but it looks like make itself precludes that so some other
> variable would have to be used at the top level.)
>
> Any insight is appreciated - this one has got me pretty stumped as to the
> "right way" to achieve this effect.
>
> Thanks,
> CY
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>

But, "-- -j${PARENT_CPUS}" is make-specific anyhow, right? So the
add_custom_target you propose only works with the Makefile generators
where the make in question actually supports -j.

In the ExternalProject module, we use the following logic to have
custom targets build where the child processes use the same parallel
job controller as the parent make process, but only for "make"
systems. To do this we use the literal "$(MAKE)" as the command to do
the actual build. And then, for non-make systems, we fall back to
"cmake --build"

    # Full code for this snippet starts at line 658 of the
Modules/ExternalProject.cmake file presently in CMake 'master' ...
    #
    # http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;h=a37771b9;hb=3ae6060f#l658
    #
    if(cfg_cmd_id STREQUAL "cmake")
      # CMake project.  Select build command based on generator.
      get_target_property(cmake_generator ${name} _EP_CMAKE_GENERATOR)
      if("${CMAKE_GENERATOR}" MATCHES "Make" AND
         ("${cmake_generator}" MATCHES "Make" OR NOT cmake_generator))
        # The project uses the same Makefile generator.  Use recursive make.
        set(cmd "$(MAKE)")
        ... code elided for brevity ...
      else()
        # Drive the project with "cmake --build".
        get_target_property(cmake_command ${name} _EP_CMAKE_COMMAND)
        if(cmake_command)
          set(cmd "${cmake_command}")
        else()
          set(cmd "${CMAKE_COMMAND}")
        endif()
        set(args --build ${binary_dir} --config ${CMAKE_CFG_INTDIR})
        ... code elided for brevity ...
      endif()
    else() # if(cfg_cmd_id STREQUAL "configure")
      # Non-CMake project.  Guess "make" and "make install" and "make test".
      # But use "$(MAKE)" to get recursive parallel make.
      set(cmd "$(MAKE)")
        ... code elided for brevity ...
    endif()

With this logic, we are able to use "make -j8" at the
parent/top-level/super- project level, and then all the child makes
that occur via "$(MAKE)" share the 8 slots for running parallel
commands as the make progresses.

Would that be a sufficient solution, or do you really need to pass
flags down for some of your custom targets based on some of the makes
that you have to deal with....?

Let us know.


Thanks,
David