(0032339)
Dimitri Merejkowsky (reporter)
2013-02-19 12:11

I've found a workaround:

setting

set(CMAKE_EXECUTABLE_FORMAT "ELF")

in the toolchain file solves the problem.

But after this, incremental builds no longer work

(0032340)
Brad King (manager)
2013-02-19 12:58

I can reproduce this with CMake 2.8.10.20130218-g0579f0 and Ninja git e758e8d0. The Unix Makefiles generator works fine.

FYI, CMakeLists.txt should not use CMAKE_FORCE_C_COMPILER. The CMakeForceCompiler module documentation states that it is for toolchain files only. Nevertheless, I can remove it from CMakeLists.txt and still reproduce the issue.

Setting CMAKE_EXECUTABLE_FORMAT to ELF in the toolchain file allows CMake to skip relinking to change RPATHs for installation and instead use its builtin ELF editor. Normally CMake detects this automatically but CMAKE_FORCE_C_COMPILER skips such detection and requires the toolchain file to set such information. Even without this issue you may still want to set it. What do you mean by "incremental builds no longer work"?

(0032341)
Dimitri Merejkowsky (reporter)
2013-02-19 13:06
edited on: 2013-02-19 13:15

> but CMAKE_FORCE_C_COMPILER skips such detection and requires the toolchain file to set such information.

Actually I did not find clear docs about what should and should not be in the toolchain file ...

Anyway, what's bothering me is that it works fine when cross-compiling using Unix Makefiles

Based on what I've seen in the source code, it looks like the detection is done twice: once for knowing whether to add a new "relink" target, and an other time when writing the cmake_install file, but I'm not sure

Also, there's this TODO:

// TODO: Add ELF check to ABI detection and get rid of
// CMAKE_EXECUTABLE_FORMAT.

maybe this is not hard to do ?

I'll be more than happy to try and ipmlement this :)

> What do you mean by "incremental builds no longer work"?

$ touch foo.cpp
$ <deploy>

-> libfoo.so is seen as up to date and is not relinked

(0032342)
Alex Neundorf (developer)
2013-02-19 14:58

More a workaround: I think when cross compiling it usually doesn't make a lot of sense to have an RPATH for build host, since you usually cannot execute the code on the build host anyway.
So how about skipping the build rpath and building directly with the install rpath ?
This should avoid the need for the relinking.

(0032343)
Brad King (manager)
2013-02-19 15:14

Re 0013934:0032341: I assigned this to Peter K specifically to investigate why the Ninja generator behaves differently from Unix Makefiles.

The TODO about CMAKE_EXECUTABLE_FORMAT, if resolved, would still not help this case because the ABI check it mentions is skipped by CMAKE_FORCE_C_COMPILER.

I cannot reproduce broken incremental builds. It works fine for me in your test case with CMAKE_EXECUTABLE_FORMAT set to ELF in the toolchain file.

(0032350)
Dimitri Merejkowsky (reporter)
2013-02-20 03:47
edited on: 2013-02-20 03:47

@Alex Neundorf: Sounds reasonable to me

@Brad King: Thanks. I'll try to write a minimal example to reproduce the
broken incremental builds.

(0034583)
ctaf (reporter)
2013-11-27 04:13

As a big hack... calling set(CMAKE_PLATFORM_HAS_INSTALLNAME 1) fix the issue. (it force using chrpath).

I added that to /usr/share/cmake-2.8/Modules/Platform/Linux-GNU.cmake to workaround the issue, til a proper fix is found.

(0034730)
LCID Fire (reporter)
2013-12-06 03:55

Is this a WONTFIX? I need cmake & ninja to build a library without any user intervention, so patching cmake is out of question.
Setting CMAKE_PLATFORM_HAS_INSTALLNAME in CMakeLists.txt also did not help.
Strangely when I generate makefiles, this breaks even more.

(0034731)
Peter Kuemmel (developer)
2013-12-06 07:52

Put in in cross.cmake only your compiler path
set(CMAKE_C_COMPILER ${MYPATH}gcc CACHE PATH "Cross C compiler" FORCE)

and remove in CMakeLists.txt
include(CMakeForceCompiler)
CMAKE_FORCE_C_COMPILER("gcc" GNU)

and it works, see ninja-solution.tar.

(0034732)
Peter Kuemmel (developer)
2013-12-06 07:57

You could also drop cross.cmake completely and just call cmake with -Dcross=1:

cmake_minimum_required(VERSION 2.8)
if(cross)
set(CMAKE_C_COMPILER ${MYPATH}gcc CACHE PATH "Cross C compiler" FORCE)
endif()
project(foo C)

be sure you set CMAKE_C_COMPILER before calling project().

(0034734)
Brad King (manager)
2013-12-06 10:02

Peter, the Ninja generator is missing logic for preinstall and NeedRelinkBeforeInstall code paths from the Makefile generators.

(0034735)
Peter Kuemmel (developer)
2013-12-06 10:05

Ah, Ok, but I never hat problems with cross-compiling.

(0034736)
Brad King (manager)
2013-12-06 10:07

Re 0013934:0034583, 0013934:0034730: One approach is that discussed in 0013934:0032339 and 0013934:0032340. By setting CMAKE_EXECUTABLE_FORMAT to "ELF" in the toolchain file you tell CMake that it can use its builtin ELF editor instead of relinking. Another approach is as suggested in 0013934:0034731 to not "force" the compiler (which bypasses the code that detects ELF).

(0034737)
Brad King (manager)
2013-12-06 10:09

Re 0013934:0034735: There are two problems under discussion here. One is that cross-compiling to an ELF target needs to be done with a toolchain file that tells CMake it is ELF in order to avoid relinking. The other is that compiling with Ninja and targeting a non-ELF platform that has RPATH (e.g. on commercial UNIX systems), whether cross-compiling or not, will not install correctly because the relinking the installation code expects to happen does not. For that the Ninja generator must implement preinstall.

(0034738)
LCID Fire (reporter)
2013-12-06 10:23

@Brad_King is the relinking only necessary because of the rpath?

(0034740)
Brad King (manager)
2013-12-06 11:16

Re 0013934:0034738: Yes. CMake by default links binaries so they can run in the build tree, so their RPATH points at build tree locations. This is not desirable for installation so CMake needs to change the RPATH to a different value (or remove it) during installation. For ELF CMake knows how to edit the binaries as they are installed. For other binary formats that support RPATH CMake needs to re-run the linker with the new RPATH options. That is done during the "preinstall" step which is a dependency of "make install" and is missing from "ninja install".

FYI, one can use CMAKE_BUILD_WITH_INSTALL_RPATH to tell CMake to put the install-tree RPATH in the build-tree binaries so that no relinking or editing is needed on installation (at the cost of not easily running in the build tree):

http://www.cmake.org/cmake/help/v2.8.12/cmake.html#variable:CMAKE_BUILD_WITH_INSTALL_RPATH [^]
http://www.cmake.org/cmake/help/v2.8.12/cmake.html#prop_tgt:BUILD_WITH_INSTALL_RPATH [^]

(0034741)
Peter Kuemmel (developer)
2013-12-06 13:54

I can confirm that setting RPATH with a ELF cross-compiler works.

(0034742)
Dimitri Merejkowsky (reporter)
2013-12-07 05:23

What if I just want to install some components ? (so calling 'make install' won't do)

Doc says I should use something like `cmake -DCOMPONENT="runtime" -Pcmake_install.cmake`

But in this case, should I call 'make preinstall' to make sure relinked libraries have been built ?

Or is there something I'm missing there ?

(0034756)
Brad King (manager)
2013-12-09 09:11

Re 0013934:0034742: Yes, you need to run "make preinstall" first in that case.

To which document do you refer?

(0034765)
Dimitri Merejkowsky (reporter)
2013-12-09 14:32

I think I only found traces of it on stack overflow, actually:

http://stackoverflow.com/questions/9190098/for-cmakes-install-command-what-can-the-component-argument-do [^]

But I also found out you already answered this very question in an old thread
http://www.cmake.org/pipermail/cmake/2006-October/011362.html [^]

:)

(0037157)
Cristian Adam (reporter)
2014-11-06 10:11

Problem reproduces also on QNX 6.5.1 on Windows using ninja.

make install works fine, while ninja install doesn't.

Adding set(CMAKE_PLATFORM_HAS_INSTALLNAME 1) to "cmake\share\cmake-2.8\Modules\Platform\QNX.cmake" fixed the problem.

(0037158)
Brad King (manager)
2014-11-06 10:20

Re 0013934:0037157: See also the workaround mentioned at the end of 0013934:0034740. That comment also explains what fix is needed upstream (implement preinstall in Ninja).

(0042228)
Kitware Robot (administrator)
2016-06-10 14:28

Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

Notes
(0032339) Dimitri Merejkowsky (reporter) 2013-02-19 12:11	I've found a workaround: setting set(CMAKE_EXECUTABLE_FORMAT "ELF") in the toolchain file solves the problem. But after this, incremental builds no longer work

(0032340) Brad King (manager) 2013-02-19 12:58	I can reproduce this with CMake 2.8.10.20130218-g0579f0 and Ninja git e758e8d0. The Unix Makefiles generator works fine. FYI, CMakeLists.txt should not use CMAKE_FORCE_C_COMPILER. The CMakeForceCompiler module documentation states that it is for toolchain files only. Nevertheless, I can remove it from CMakeLists.txt and still reproduce the issue. Setting CMAKE_EXECUTABLE_FORMAT to ELF in the toolchain file allows CMake to skip relinking to change RPATHs for installation and instead use its builtin ELF editor. Normally CMake detects this automatically but CMAKE_FORCE_C_COMPILER skips such detection and requires the toolchain file to set such information. Even without this issue you may still want to set it. What do you mean by "incremental builds no longer work"?

(0032341) Dimitri Merejkowsky (reporter) 2013-02-19 13:06 edited on: 2013-02-19 13:15	> but CMAKE_FORCE_C_COMPILER skips such detection and requires the toolchain file to set such information. Actually I did not find clear docs about what should and should not be in the toolchain file ... Anyway, what's bothering me is that it works fine when cross-compiling using Unix Makefiles Based on what I've seen in the source code, it looks like the detection is done twice: once for knowing whether to add a new "relink" target, and an other time when writing the cmake_install file, but I'm not sure Also, there's this TODO: // TODO: Add ELF check to ABI detection and get rid of // CMAKE_EXECUTABLE_FORMAT. maybe this is not hard to do ? I'll be more than happy to try and ipmlement this :) > What do you mean by "incremental builds no longer work"? $ touch foo.cpp $ <deploy> -> libfoo.so is seen as up to date and is not relinked

(0032342) Alex Neundorf (developer) 2013-02-19 14:58	More a workaround: I think when cross compiling it usually doesn't make a lot of sense to have an RPATH for build host, since you usually cannot execute the code on the build host anyway. So how about skipping the build rpath and building directly with the install rpath ? This should avoid the need for the relinking.

(0032343) Brad King (manager) 2013-02-19 15:14	Re 0013934:0032341: I assigned this to Peter K specifically to investigate why the Ninja generator behaves differently from Unix Makefiles. The TODO about CMAKE_EXECUTABLE_FORMAT, if resolved, would still not help this case because the ABI check it mentions is skipped by CMAKE_FORCE_C_COMPILER. I cannot reproduce broken incremental builds. It works fine for me in your test case with CMAKE_EXECUTABLE_FORMAT set to ELF in the toolchain file.

(0032350) Dimitri Merejkowsky (reporter) 2013-02-20 03:47 edited on: 2013-02-20 03:47	@Alex Neundorf: Sounds reasonable to me @Brad King: Thanks. I'll try to write a minimal example to reproduce the broken incremental builds.

(0034583) ctaf (reporter) 2013-11-27 04:13	As a big hack... calling set(CMAKE_PLATFORM_HAS_INSTALLNAME 1) fix the issue. (it force using chrpath). I added that to /usr/share/cmake-2.8/Modules/Platform/Linux-GNU.cmake to workaround the issue, til a proper fix is found.

(0034730) LCID Fire (reporter) 2013-12-06 03:55	Is this a WONTFIX? I need cmake & ninja to build a library without any user intervention, so patching cmake is out of question. Setting CMAKE_PLATFORM_HAS_INSTALLNAME in CMakeLists.txt also did not help. Strangely when I generate makefiles, this breaks even more.

(0034731) Peter Kuemmel (developer) 2013-12-06 07:52	Put in in cross.cmake only your compiler path set(CMAKE_C_COMPILER ${MYPATH}gcc CACHE PATH "Cross C compiler" FORCE) and remove in CMakeLists.txt include(CMakeForceCompiler) CMAKE_FORCE_C_COMPILER("gcc" GNU) and it works, see ninja-solution.tar.

(0034732) Peter Kuemmel (developer) 2013-12-06 07:57	You could also drop cross.cmake completely and just call cmake with -Dcross=1: cmake_minimum_required(VERSION 2.8) if(cross) set(CMAKE_C_COMPILER ${MYPATH}gcc CACHE PATH "Cross C compiler" FORCE) endif() project(foo C) be sure you set CMAKE_C_COMPILER before calling project().

(0034734) Brad King (manager) 2013-12-06 10:02	Peter, the Ninja generator is missing logic for preinstall and NeedRelinkBeforeInstall code paths from the Makefile generators.

(0034735) Peter Kuemmel (developer) 2013-12-06 10:05	Ah, Ok, but I never hat problems with cross-compiling.

(0034736) Brad King (manager) 2013-12-06 10:07	Re 0013934:0034583, 0013934:0034730: One approach is that discussed in 0013934:0032339 and 0013934:0032340. By setting CMAKE_EXECUTABLE_FORMAT to "ELF" in the toolchain file you tell CMake that it can use its builtin ELF editor instead of relinking. Another approach is as suggested in 0013934:0034731 to not "force" the compiler (which bypasses the code that detects ELF).

(0034737) Brad King (manager) 2013-12-06 10:09	Re 0013934:0034735: There are two problems under discussion here. One is that cross-compiling to an ELF target needs to be done with a toolchain file that tells CMake it is ELF in order to avoid relinking. The other is that compiling with Ninja and targeting a non-ELF platform that has RPATH (e.g. on commercial UNIX systems), whether cross-compiling or not, will not install correctly because the relinking the installation code expects to happen does not. For that the Ninja generator must implement preinstall.

(0034738) LCID Fire (reporter) 2013-12-06 10:23	@Brad_King is the relinking only necessary because of the rpath?

(0034740) Brad King (manager) 2013-12-06 11:16	Re 0013934:0034738: Yes. CMake by default links binaries so they can run in the build tree, so their RPATH points at build tree locations. This is not desirable for installation so CMake needs to change the RPATH to a different value (or remove it) during installation. For ELF CMake knows how to edit the binaries as they are installed. For other binary formats that support RPATH CMake needs to re-run the linker with the new RPATH options. That is done during the "preinstall" step which is a dependency of "make install" and is missing from "ninja install". FYI, one can use CMAKE_BUILD_WITH_INSTALL_RPATH to tell CMake to put the install-tree RPATH in the build-tree binaries so that no relinking or editing is needed on installation (at the cost of not easily running in the build tree): http://www.cmake.org/cmake/help/v2.8.12/cmake.html#variable:CMAKE_BUILD_WITH_INSTALL_RPATH [^] http://www.cmake.org/cmake/help/v2.8.12/cmake.html#prop_tgt:BUILD_WITH_INSTALL_RPATH [^]

(0034741) Peter Kuemmel (developer) 2013-12-06 13:54	I can confirm that setting RPATH with a ELF cross-compiler works.

(0034742) Dimitri Merejkowsky (reporter) 2013-12-07 05:23	What if I just want to install some components ? (so calling 'make install' won't do) Doc says I should use something like `cmake -DCOMPONENT="runtime" -Pcmake_install.cmake` But in this case, should I call 'make preinstall' to make sure relinked libraries have been built ? Or is there something I'm missing there ?

(0034756) Brad King (manager) 2013-12-09 09:11	Re 0013934:0034742: Yes, you need to run "make preinstall" first in that case. To which document do you refer?

(0034765) Dimitri Merejkowsky (reporter) 2013-12-09 14:32	I think I only found traces of it on stack overflow, actually: http://stackoverflow.com/questions/9190098/for-cmakes-install-command-what-can-the-component-argument-do [^] But I also found out you already answered this very question in an old thread http://www.cmake.org/pipermail/cmake/2006-October/011362.html [^] :)

(0037157) Cristian Adam (reporter) 2014-11-06 10:11	Problem reproduces also on QNX 6.5.1 on Windows using ninja. make install works fine, while ninja install doesn't. Adding set(CMAKE_PLATFORM_HAS_INSTALLNAME 1) to "cmake\share\cmake-2.8\Modules\Platform\QNX.cmake" fixed the problem.

(0037158) Brad King (manager) 2014-11-06 10:20	Re 0013934:0037157: See also the workaround mentioned at the end of 0013934:0034740. That comment also explains what fix is needed upstream (implement preinstall in Ninja).

(0042228) Kitware Robot (administrator) 2016-06-10 14:28	Resolving issue as `moved`. This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

Issue History
Date Modified	Username	Field	Change
2013-02-19 12:07	Dimitri Merejkowsky	New Issue
2013-02-19 12:07	Dimitri Merejkowsky	File Added: ninjabug.tar.gz
2013-02-19 12:11	Dimitri Merejkowsky	Note Added: 0032339
2013-02-19 12:58	Brad King	Note Added: 0032340
2013-02-19 12:58	Brad King	Assigned To	=> Peter Kuemmel
2013-02-19 12:58	Brad King	Status	new => assigned
2013-02-19 13:06	Dimitri Merejkowsky	Note Added: 0032341
2013-02-19 13:15	Dimitri Merejkowsky	Note Edited: 0032341
2013-02-19 14:58	Alex Neundorf	Note Added: 0032342
2013-02-19 15:14	Brad King	Note Added: 0032343
2013-02-20 03:47	Dimitri Merejkowsky	Note Added: 0032350
2013-02-20 03:47	Dimitri Merejkowsky	Note Edited: 0032350
2013-11-27 04:13	ctaf	Note Added: 0034583
2013-12-06 03:55	LCID Fire	Note Added: 0034730
2013-12-06 07:48	Peter Kuemmel	Assigned To	Peter Kuemmel =>
2013-12-06 07:49	Peter Kuemmel	File Added: ninja-solution.tar
2013-12-06 07:52	Peter Kuemmel	Note Added: 0034731
2013-12-06 07:57	Peter Kuemmel	Note Added: 0034732
2013-12-06 10:02	Brad King	Note Added: 0034734
2013-12-06 10:02	Brad King	Assigned To	=> Peter Kuemmel
2013-12-06 10:05	Peter Kuemmel	Note Added: 0034735
2013-12-06 10:05	Peter Kuemmel	Status	assigned => backlog
2013-12-06 10:07	Brad King	Note Added: 0034736
2013-12-06 10:09	Brad King	Note Added: 0034737
2013-12-06 10:23	LCID Fire	Note Added: 0034738
2013-12-06 11:16	Brad King	Note Added: 0034740
2013-12-06 13:54	Peter Kuemmel	Note Added: 0034741
2013-12-07 05:23	Dimitri Merejkowsky	Note Added: 0034742
2013-12-09 09:11	Brad King	Note Added: 0034756
2013-12-09 14:32	Dimitri Merejkowsky	Note Added: 0034765
2014-11-06 10:11	Cristian Adam	Note Added: 0037157
2014-11-06 10:20	Brad King	Note Added: 0037158
2016-06-10 14:28	Kitware Robot	Note Added: 0042228
2016-06-10 14:28	Kitware Robot	Status	backlog => resolved
2016-06-10 14:28	Kitware Robot	Resolution	open => moved
2016-06-10 14:31	Kitware Robot	Status	resolved => closed

Relationships