[CMake] Trying to set up a "superbuild" with external project and install it...

[Michael Hertling]

On 05/14/2011 07:57 PM, Alexander Neundorf wrote:
> On Thursday 12 May 2011, Michael Hertling wrote:
>> On 05/11/2011 09:07 PM, Alexander Neundorf wrote:
>>> Hi,
>>>
>>> I'd like to set up a kind of meta super project, which builds a bunch of
>>> subprojects, which contains executable and shared libraries.
>>> These subprojects come from different repositories, mostly git
>>> repositories.
>>>
>>> AFAIK I can basically do this using the ExternalProject_add() feature of
>>> cmake, it will get the sources, configure and build them.
>>>
>>> So far so good.
>>>
>>>
>>> Now, what do I do with installing ?
>>> Usually the external projects are installed at the end of their build
>>> step at make-time, and usually somewhere in the build tree.
>>>
>>> But finally I'd like to have these projects installed into a proper
>>> system- wide directory.
>>>
>>> If I simply do install(DIRECTORY ... ) in the "superproject", stuff which
>>> depends on CMAKE_INSTALL_PREFIX, like e.g. RPATH will be wrong after the
>>> final install step (since the RPATHs will still point to the install
>>> locations in the build tree).
>>>
>>> If I put the final install location as install location in the
>>> ExternalProject_Add() call, I will have to build with permissions to
>>> install into the final install directory, which is generally not the
>>> case. E.g. to install in /opt/ I would basically have to build as root
>>> (or change the permissions of /opt/, but that's not possible on a
>>> standard system).
>>>
>>> I could maybe also setuid cmake root (ugly too).
>>>
>>> Always set a relative RPATH always using $ORIGIN ?
>>> (but this will be kind of hard to enforce)
>>>
>>> Any other ideas ?
>>> Am I missing something ?
>>> Any ideas how support for this could be added in a clean way ?
>>>
>>> Alex
>>
>> A quite similar topic has been touched on recently in [1]. My personal
>> opinion on this particular issue with regard to subprojects is roughly:
>>
>> 1. In the end, each project - subproject or not - must be installed at
>> the location it has been configured for; this is the only bullet-proof
>> way to ensure that things like RPATH or incorporated paths derived from
>> the installation prefix, e.g. <prefix>/etc or /etc if <prefix> is /usr,
>> will work as expected, or in other words: Relocation after installation
>> is critical, unless it's taken into account by any appropriate measures.
>>
>> 2. For the, say, intermediate installation of a subproject in the build
>> tree of the superproject, I'd suggest to use the DESTDIR feature, i.e.:
>>
>> - Configure and build the subproject with its final installation prefix.
>> - Install with "make DESTDIR=${CMAKE_BINARY_DIR}/externals" or the like
>>   with the superproject's CMAKE_BINARY_DIR. Thereafter, you could call
>>   FIND_PACKAGE() on the subproject with CMAKE_PREFIX_PATH and friends
>>   set to ${CMAKE_BINARY_DIR}/externals/${CMAKE_INSTALL_PREFIX}[/...]
>>   which may be important for the actual main project. Nevertheless,
>>   the subproject's binaries might not run if they strictly require
>>   to reside at the location they've been configured for.
>> - At the superproject's installation, use
>>
>> INSTALL(DIRECTORY ${CMAKE_BINARY_DIR}/externals/${CMAKE_INSTALL_PREFIX}/
>>         DESTINATION ${CMAKE_INSTALL_PREFIX})
>>
>>   to relocate the subprojects to their configured installation prefix.
> 
> 
> Yes, this work pretty good.
> But it still has a problem.
> Assume subproject foo installs a shared library libfoo, amd subprojects foo-
> bar and foo-blub are plugins for this shared library.
> 
> foo-bar and foo-blub will find libfoo.so in CMAKE_BINARY_DIR/Install/, and 
> link against it happily.
> 
> But they will have a problem with their RPATH.
> If they use e.g. ${FOO_LIBRARY_DIR} as RPATH, this would point to the 
> temporary install location of libfoo, not the final one.
> If they use INSTALL_RPATH_USE_LINK_PATH they will have the same problem.
> 
> I guess I have to install the subprojects to their final destination already 
> in the install-step of the external project.
> 
> Alex

Indeed, RPATH-related dependencies among the intermediately installed
subprojects are malicious. In order to solve the problem you outlined
without doing the whole stuff as root, you would need to install foo
to its final destination and reconfigure/rebuild foo-{bar,blub} with
CMAKE_PREFIX_PATH set to CMAKE_INSTALL_PREFIX before they're finally
installed, too - another example for the worst case. This could be
achieved by an own installation target, say, "superinstall", e.g.:

ADD_CUSTOM_TARGET(superinstall
    COMMAND ${CMAKE_COMMAND} -P cmake_install.cmake
    COMMAND <reconfigure/rebuild foo-bar with CMAKE_PREFIX_PATH=...>
    COMMAND <reconfigure/rebuild foo-blub with CMAKE_PREFIX_PATH=...>
    COMMAND ${CMAKE_COMMAND} -P cmake_install.cmake)

The first command installs the superproject including the subprojects
with the wrong RPATHs to their final location, the following commands
reconfigure/rebuild the affected subprojects as far as necessary, and
the final command reinstalls the superproject with correct RPATHs in
the subprojects. If there are chained or cascaded dependencies among
the subprojects, e.g. foo-blub incorporating ${FOO-BAR_LIBRARY_PATH}
in its RPATHs, one would need to intersperse the cmake_install.cmake
script more often. In order to prevent that the reconfigure/rebuild
steps are run with root privileges, one could add, e.g.,

SET(BUILD_USER $ENV{USER} CACHE STRING "...")

to the superproject's CMakeLists.txt and write the reconfigure/rebuild
commands as "COMMAND su -c '<reconfigure/rebuild ...>' ${BUILD_USER}",
provided the superproject is primarily targeted at *nix systems.

At the end of the day, this entire issue seems to boil down to the
question if one is willing either to rebuild possibly considerable
parts of the project during the installation or to acquire root's
privileges early to build the project - or to fragment and intermix
the build/installation process: make foo; su -c 'make install_foo';
make foo-bar foo-blub; su -c 'make install_foo-bar install_foo-blub'
etc. Apparently, that's the price of non-trivial superbuild set-ups.
Sometimes, I think they lead to more problems than they solve... ;-)

An interesting option is the manipulation of the binaries' RPATH just
before or after their final installation, similar to CMake's own post-
treatment of freshly installed libraries/executables. To do this, one
might use utilities like the obviously ceased chrpath or the actively
developed PatchELF in association with INSTALL(CODE|SCRIPT ...) or a
custom "preinstall" target. However, this would take care of RPATHs
but not other paths incorporated in the binaries and derived from
results of FIND_PACKAGE() for the intermediate installations.

If you actually decide to install the subprojects immediately to their
final destinations, I would suggest to think about the above-mentioned
BUILD_USER variable or the like and, as the case may be, encompass the
configure/build commands in the ExternalProject invocations by "su -c
'...' ${BUILD_USER}" to ensure that only the pure installation steps
are performed with root's privileges. Otherwise, using "su -c make"
to build the project means changing the environment and possibly
even the shell, so there might be additional issues to consider.

Regards,

Michael