[CMake] FYI - From Ninja-build mailing list - Fwd: Proposal: restat rules

[Andreas Mohr]

Hi,

On Wed, Sep 14, 2011 at 12:00:05PM -0400, cmake-request at cmake.org wrote:
> Date: Wed, 14 Sep 2011 05:37:20 -0400
> From: Clifford Yapp <cliffyapp at gmail.com>
> 
> Looks like that's working.  Running ninja again, I'm seeing another issue:
> 
> BRL-CAD uses dependency assignment to make sure our build time delta
> calculator is the last target to be built (and hence actually times the
> build).  With ninja, it doesn't seem to be respecting this, but instead
> tries to run the delta target immediately.  Do custom targets respect
> dependency information?

I'm seeing the same thing here, and I think I semi-nailed it.

I've got:

function(scm_sos_pull_db_tree _pull_target _scm_dir)
  if(NOT TARGET scm_sos_pull_db_tree)
    # this crap does not work the way it should...
    #set(scm_spec "${scm_repo_prefix}/${_scm_dir}")
    set(scm_spec "${scm_repo_prefix}")

    #set(stamp_file_tree "${sos_wrapper_stamps_dir}/scm_sos_pull_db_tree_${_pull_target}.stamp")
    set(stamp_file_tree "${sos_wrapper_stamps_dir}/scm_sos_pull_db_tree.stamp")
    set(db_tree_output "${sos_wrapper_db_file_pathname}")
    add_custom_command(OUTPUT "${db_tree_output}"
      COMMAND ${sos_wrapper_cmdline_constant_part} "-project ${scm_spec} -command GetProjectTree"
      COMMAND "${CMAKE_COMMAND}" -E touch "${stamp_file_tree}"
      # Cannot specify the full ${scm_sos_dependencies} list here
      # (we're creating the very database file that it lists).
      # And we should avoid adding ${sos_wrapper_script} as a dependency
      # here, since a simple update of that script most certainly doesn't
      # justify doing the _AWFULLY_ lengthy operation of pulling
      # the entire database anew. All other SCM targets _should_ properly depend on
      # SosWrap.bash, however, since they're much cheaper...
      DEPENDS "${MASTER_SCM_HAS_BEEN_UPDATED}"
      COMMENT "fetching SOS SCM database tree for ${scm_spec}"
      VERBATIM
    )
    add_custom_target(scm_sos_pull_db_tree VERBATIM DEPENDS "${db_tree_output}")
    add_dependencies(scm_sos_pull_db_tree scm_sos_setup)
  endif(NOT TARGET scm_sos_pull_db_tree)
  # add a dependency to make sure that before pulling a file/project we fetch its project tree
  add_dependencies(${_pull_target} scm_sos_pull_db_tree)
endfunction(scm_sos_pull_db_tree _pull_target _scm_dir)


So you can see that I'm clearly adding a dependency of target scm_sos_pull_db_tree on the
target scm_sos_setup (which is the one to make sure that the SosWrap.bash script
is properly prepared before continuing).


However, what I'm ending up with is:



# Phony custom command for CMakeFiles/scm_sos_pull_db_tree
build CMakeFiles/scm_sos_pull_db_tree: phony _SCM/sos/database/servers/SERVER/DATABASE.sos
# Utility command for scm_sos_pull_db_tree
build scm_sos_pull_db_tree: phony CMakeFiles/scm_sos_pull_db_tree _SCM/sos/database/servers/SERVER/DATABASE.sos scm_sos_setup
# Phony custom command for CMakeFiles/scm_sos_setup
build CMakeFiles/scm_sos_setup: phony _SCM/sos/stamps/sos_setup.stamp
# Custom command for _SCM/sos/stamps/sos_setup.stamp
build _SCM/sos/stamps/sos_setup.stamp: CUSTOM_COMMAND
  COMMAND = cd [[CMAKE_BINARY_DIR]] && /usr/local/bin/cmake -E make_directory [[CMAKE_BINARY_DIR]]/_SCM/sos/stamps && /usr/local/bin/cmake -E touch [[CMAKE_BINARY_DIR]]/_SCM/sos/stamps/sos_setup.stamp
  DESC = Generating _SCM/sos/stamps/sos_setup.stamp
# Utility command for scm_sos_setup
build scm_sos_setup: phony CMakeFiles/scm_sos_setup _SCM/sos/stamps/sos_setup.stamp copy_prep_file__[[CMAKE_BINARY_DIR]]_SosWrap.bash


IOW, from what I'm gathering here, it seems as if   scm_sos_pull_db_tree   lists   its _internal_
target/rule/whatever CMakeFiles/scm_sos_pull_db_tree with same-hierarchy priority as scm_sos_setup.
And that internal target/rule/whatever then gets executed in advance, despite scm_sos_setup not having been executed (prepared) yet.

And this is most likely because add_custom_command() in Ninja generator
actually gets realised as an inner target of its corresponding CMake add_custom_target(),
and we then have a logical disconnect since it's the _outer_, CMake-_public_ target
which gets configured a constraint ("scm_sos_setup needs to run first").


IOW, it's a MISTAKE to implement (emulate) add_custom_command() via inner targets in Ninja build config,
since those inner targets don't inherit those dependency constraints ("ordering")
which have been explicitly imposed on their outer targets.

So:
- either keep custom commands implemented via inner targets - and then make sure
  that those inner (internal!) targets do have _all_ dependencies of the outer,
  CMake-public target specified
- or don't implement custom commands via internal, logically disconnected targets  [better?]


Or am I missing something?
(I believe my CMake construct is correct - but who knows...)

OK, I should perhaps add a file dependency (DEPENDS) within add_custom_command() on the
SosWrap.bash script (but that was done _very_ deliberately - see comment in that function() above,
and the purpose of that DEPENDS is usually solely to trigger a _rebuild_
in case the file dependency changed, and that does work properly with Unix Makefiles
- actual execution _ordering_ is supposed to be done by specification of corresponding
target dependencies, and this is what fails with Ninja generator).


Ninja generator - blazingly fast execution due to guillotineing some 72.13% of all dependencies? ;->
(whoa, that perhaps was almost as severe as an ad-hominem ;)



Thanks a lot again for your very valuable work!

Andreas Mohr