[CMake] External projects and make clean

[Michael Hertling]

On 02/17/2012 10:16 AM, Oliver Boesche wrote:
> Hi,
> 
> I use external projects e.g. to build BOOST for my own project. In that 
> case I want to prevent the project to be system or version dependent.
> 
> So I build my dependencies successful with external project (and its 
> great), but if use make clean it will clean all stuff and I have to 
> rebuild the external projects again (take some time with BOOST).
> 
> Is there a solution to prevent an external project from cleaning when I 
> call 'make clean'?
> 
> Kind regards
> 
> Oliver Boesche

For the Makefile generators, I could offer the following approach:

# CMakeLists.txt:
CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
PROJECT(P C)
SET(CMAKE_VERBOSE_MAKEFILE ON)

INCLUDE(ExternalProject)

ExternalProject_Add(external
    SOURCE_DIR ${CMAKE_SOURCE_DIR}/external
    CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
)

FILE(WRITE ${CMAKE_BINARY_DIR}/main.c "int main(void){return 0;}\n")
ADD_EXECUTABLE(main main.c)
ExternalProject_Get_Property(external INSTALL_DIR)
TARGET_LINK_LIBRARIES(main ${INSTALL_DIR}/lib/libf.so)

ExternalProject_Get_Property(external STAMP_DIR)

ADD_CUSTOM_TARGET(restorestampfiles
    COMMAND ${CMAKE_COMMAND}
        -DSTAMP_DIR="${STAMP_DIR}"
        -P "${CMAKE_SOURCE_DIR}/restorestampfiles.cmake"
    COMMENT "Restoring stamp files")

ADD_CUSTOM_TARGET(backupstampfiles
    COMMAND ${CMAKE_COMMAND}
        -DSTAMP_DIR="${STAMP_DIR}"
        -P "${CMAKE_SOURCE_DIR}/backupstampfiles.cmake"
    COMMENT "Backing up stamp files")

ADD_CUSTOM_TARGET(toplevel-clean
    COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR}
                             --target backupstampfiles
    COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR}
                             --target clean
    COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR}
                             --target restorestampfiles)

# external/CMakeLists.txt:
CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
PROJECT(EXTERNAL C)
SET(CMAKE_VERBOSE_MAKEFILE ON)
FILE(WRITE ${CMAKE_BINARY_DIR}/f.c "void f(void){}\n")
ADD_LIBRARY(f SHARED f.c)
INSTALL(TARGETS f DESTINATION lib)

# backupstampfiles.cmake:
FILE(GLOB_RECURSE STAMP_FILES "${STAMP_DIR}/*")
FOREACH(i IN LISTS STAMP_FILES)
    IF(NOT i MATCHES "\\.bak$" AND NOT IS_DIRECTORY "${i}")
        FILE(RENAME "${i}" "${i}.bak")
    ENDIF()
ENDFOREACH()

# restorestampfiles.cmake:
FILE(GLOB_RECURSE STAMP_FILES "${STAMP_DIR}/*")
FOREACH(i IN LISTS STAMP_FILES)
    IF(i MATCHES "\\.bak$")
        STRING(REGEX REPLACE "\\.bak$" "" j "${i}")
        FILE(RENAME "${i}" "${j}")
    ENDIF()
ENDFOREACH()

The basic idea is to backup the external project's stamp files before
cleaning, and restore them afterwards. In this way, the steps of the
external project aren't performed during the next build. As a proof,
configure and build the above-noted exemplary project, then issue:

make clean; make

You'll see that the external and the toplevel project are both rebuild
although the external one's binary survives the cleaning. Now, issue:

make toplevel-clean; make

This time, the external project's steps are left out, and only the
toplevel project is rebuilt. In order to be absolutely sure, remove
the ${CMAKE_BINARY_DIR}/external-prefix/lib/libf.so library file to
see "make toplevel-clean; make" fail, whereas "make clean; make" re-
builds the missing file and, thus, succeeds.

Regrettably, the targets "restorestampfiles" and "backupstampfiles"
can not be turned into a custom step of the external project:

ExternalProject_Add_Step(external restore
    COMMAND ${CMAKE_COMMAND} ... -P ...restorestampfiles.cmake
    DEPENDERS mkdir)

ExternalProject_Add_Step(external backup
    COMMAND ${CMAKE_COMMAND} ... -P ...backupstampfiles.cmake
    DEPENDEES install)

This would be quite elegant, but apparently, the out-of-dateness of the
external project's steps is recognized before the stamp files are re-
stored. Moreover, I didn't manage to make this appoach work with VS.

IMO, the principal question in this regard is: What do we expect from
the "clean" target w.r.t. external projects? Usually, the latters are
not subject to the toplevel project's development, so one would like
to build them once, but omit them from cleaning. OTOH, there must be
a possibility to clean the toplevel project along with the external
ones in order to perform a full rebuild. AFAICS, the EP module does
not support both variants. A conceptual solution I can imagine is:

- Add a "clean" step - running a CLEAN_COMMAND, possibly defaulting
  to "make clean" - to the external project which is performed when
  the toplevel project is cleaned.

- Add a flag, say, EXCLUDE_FROM_CLEAN that, when specified, excludes
  the external project from the toplevel project's "clean" target.

This EXCLUDE_FROM_CLEAN flag in conjunction with the STEP_TARGETS option
of ExternalProject_Add() would allow for the definition of fine-grained
"clean" targets for external projects. E.g.,

External_Project_Add(boost
    ...
    EXCLUDE_FROM_CLEAN
    ...
    STEP_TARGETS clean ....)

ADD_CUSTOM_TARGET(total-clean
    ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR} --target clean)

ADD_DEPENDENCIES(total-clean boost-clean ...)

would exclude Boost from the usual "clean" target, provide a target
"boost-clean" that cleans Boost only, and finally a target "total-
clean" which cleans the toplevel project along with the external
ones. Perhaps, this might be feasible with an acceptable effort.

Regards,

Michael