MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0014707CMakeCMakepublic2014-01-17 15:402014-10-06 10:32
ReporterFrancis Chan 
Assigned ToClinton Stimpson 
PrioritynormalSeverityblockReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0014707: Install target command corrupts binary during RPATH modification
DescriptionBinary is compiled with OSX Platform 2.7
CMakelist does not set MACOSX_RPATH property
Cmakelist does not make use of the @RPATH (unless set by default in 2.8.12.1)
Binary is installed using INSTALL( TARGET RUNTIME DESTINATION ... ).
CMake 2.8.11 does not corrupt binary.
If corruption occurs on a binary, the corruption will always happen for that given binary. May be related to how cmake determines what paths to strip and what are the dependencies of the binary. Some artifacts will install correctly.
Steps To ReproduceVery hard to reproduce since we don't know what cmake does during install or what specifically in the dependencies or binary which causes the install to go awry.

We tried to replicate the install_name_tool commands used but they do not corrupt the binary. We are assuming something else is being done during install but not sure how to get more information on what exactly the install command does (besides run the install_name_tool for RPATH strip).
Additional InformationLogs during installation shows install_name_tool reporting corruption:
     [exec] -- Installing: <DELETED_APPLICATION_NAME>
     [exec] /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: no LC_RPATH load command with path: <DELETED_PATH> found in: <DELETED_APPLICATION_NAME> (for architecture i386), required for specified option "-delete_rpath <DELETED_PATH>"
     [exec] /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: object: <DELETED_APPLICATION_NAME> malformed object (load command 52 cmdsize is zero)


Binary that gets corrupted has the following dependency list:
    /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 17.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 635.21.0)
    /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 53.0.0)
    /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
    /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 395.11.0)
    /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 7.0.0)
    /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
    /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 833.25.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

Binary that passes install has the following dependency list:
    /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 17.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 635.21.0)
    /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 53.0.0)
    /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
    /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 395.11.0)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
    /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
TagsNo tags attached.
Relationships
Attached Filestxt CMakeLists.txt (673) 2014-01-25 14:42
https://public.kitware.com/Bug/file/5051/CMakeLists.txt
Issue History
Date ModifiedUsernameFieldChange
2014-01-17 15:40Francis ChanNew Issue
2014-01-17 15:54Brad KingAssigned To => Clinton Stimpson
2014-01-17 15:54Brad KingStatusnew => assigned
2014-01-17 15:56Clinton StimpsonNote Added: 0034956
2014-01-17 16:06Francis ChanNote Added: 0034957
2014-01-17 16:53Clinton StimpsonNote Added: 0034958
2014-01-17 20:36Francis ChanNote Added: 0034959
2014-01-17 21:35Clinton StimpsonNote Added: 0034960
2014-01-18 12:28Clinton StimpsonNote Added: 0034961
2014-01-18 15:17Clinton StimpsonNote Added: 0034962
2014-01-20 14:15Clinton StimpsonNote Added: 0034977
2014-01-20 14:20Brad KingNote Added: 0034978
2014-01-20 22:23Francis ChanNote Added: 0034980
2014-01-22 20:39Francis ChanNote Added: 0034988
2014-01-22 23:06Clinton StimpsonNote Added: 0034989
2014-01-24 19:02Francis ChanNote Added: 0034993
2014-01-25 14:42Clinton StimpsonFile Added: CMakeLists.txt
2014-01-25 15:00Clinton StimpsonNote Added: 0034994
2014-01-27 09:18Brad KingNote Added: 0034995
2014-01-27 11:18Clinton StimpsonNote Added: 0035002
2014-01-27 11:21Brad KingNote Added: 0035003
2014-02-01 23:25Clinton StimpsonNote Added: 0035018
2014-02-01 23:25Clinton StimpsonStatusassigned => resolved
2014-02-01 23:25Clinton StimpsonResolutionopen => fixed
2014-10-06 10:32Robert MaynardNote Added: 0036903
2014-10-06 10:32Robert MaynardStatusresolved => closed

Notes
(0034956)
Clinton Stimpson   
2014-01-17 15:56   
Which OS X version and Xcode version is this?
I've seen problems with older versions of install_name_tool corrupting binaries.
The corruption was avoided by doing rpath modifications one at a time instead of in bulk.

Do you have an example to demonstrate the problem?
(0034957)
Francis Chan   
2014-01-17 16:06   
OSX 10.7.5
XCode 4.6

We have the binary but it contains proprietary code so I can't release that for test.

We may be able to alter the binary to maintain the dependencies but exclude any company proprietary data but I will need to follow up with engineering on what is the ETA to do this while ensuring the modifications still reproduces the corruption.

We know that 2.8.11 does not show this issue (same toolchain, different cmake).
(0034958)
Clinton Stimpson   
2014-01-17 16:53   
Are you using either
CMAKE_MACOSX_RPATH
or
link_directories()
in your project?
(0034959)
Francis Chan   
2014-01-17 20:36   
link_directories( )
(0034960)
Clinton Stimpson   
2014-01-17 21:35   
Does removing uses of link_directories() solve your problem? I found a bug today where the use of link_directories() will add undesired rpaths.

Anyway, using link_directories() has been discouraged for quite some time.

If that fixes your problem, that still doesn't help us find out why the binaries are corrupted.
(0034961)
Clinton Stimpson   
2014-01-18 12:28   
Also, the rpath modifications are done by cmake_install.cmake found in the build directory and sub-directories.

Can you see what commands it is doing and narrow the problem down?
(0034962)
Clinton Stimpson   
2014-01-18 15:17   
This commit may fix your problem of a build that broke with a CMake update.
http://cmake.org/gitweb?p=cmake.git;a=commit;h=b9e8ff4 [^]

However, it would still be wonderful if you could narrow down the corruption problem.
I suspect it is a problem in install_name_tool that should be reported to Apple.
(0034977)
Clinton Stimpson   
2014-01-20 14:15   
After some discussion, the commit was reverted, and we do want link_directories() to add rpaths.

Your options now are:
1. remove use of link_directories()
2. help us narrow down the problem
(0034978)
Brad King   
2014-01-20 14:20   
Re 0014707:0034957: If you can't reproduce it in a public test case but know it worked in 2.8.11 then perhaps you can help us find the change between 2.8.11 and 2.8.12 that introduced the problem. Can you build CMake from source with

 git clone http://cmake.org/cmake.git [^]

and then use 'git bisect' to narrow down the change?
(0034980)
Francis Chan   
2014-01-20 22:23   
Ok I'll need a little time with this since I got swamped with some emergency @ work. I'll first take a look at the cmake_install.cmake to get a feel for the work, then remove the link_directories(), then move to the github build/diff log and see how those options pan out.

I have this on my list for this week so give me a few days to squeeze some time in to get you more details.
(0034988)
Francis Chan   
2014-01-22 20:39   
I've inspected the cmake_install.cmake, ran some scenarios, and have been able to reproduce the failure now. I believe the problem is related to install_name_tool failing to properly strip an LC_RPATH command if the same command appears twice in the binary. This is probably also related to how link_directories() in cmake 2.8.12 determines whether or not a path should be inserted into the RPATH (or something along that line).

1. It's related to link_directories()
2. It happens when two link_directories( ) commands are used, resolve to the same final path, but use _different_ macro expansion strings. The binary subsequently has subsequently has two identical LC_RPATHs.
3. Based on what I've tried (without digging around in the code), link_directories( ) is smart enough _not_ to include a directory twice but that comparison is done pre-expansion, not post.

I can solve this problem by cleaning up this link_directories() commands. Also, not using link_directories, solves this issue as well. Our newer build files no longer use link_directories.
(0034989)
Clinton Stimpson   
2014-01-22 23:06   
Ah... yep, I see the problem now.

The only way I can make a binary with duplicate LC_RPATHs is to pass the duplication with linker flags.
install_name_tool will refuse to add duplicate paths, but install_name_tool will corrupt the binary when removing if there is a duplicate.

Can you explain the macro expansion strings you were referring to?
I tried this
set(path1 /usr/local/lib)
set(path2 /usr/local/lib)
link_directories(${path1})
link_directories(${path2})
and was not able to see the duplication of "/usr/local/lib"

Or ideally, are you now able to produce a small test case showing the problem?
(0034993)
Francis Chan   
2014-01-24 19:02   
I did a little more experimentation and should say that it's not a CMake macro expansion problem after a little more careful testing.

This is really confusing so I'll try to explain the best way I can.

For whatever reason, we use $(CONFIGURATION) as instead of ${CMAKE_BUILD_TYPE} for the OSX target. That flag is expanded at the time of compilation by the linker/compiler.

Ex.:

(set P1 $(CONFIGURATION))
(set P2 Release)
link_directories(${P1})
link_directories(${P2})

This will result in cmake thinking the first link_directories is "$(CONFIGURATION)" and the second as "Release". To CMake they appear to be two different paths but to the linker they are identical when it begins to link the Release build. This is how two identical paths are being injected into the LC_RPATH.

In fact, I can see in when the install.cmake gets executed, install_name_tool keeps complaining about all these paths with $(CONFIGURATION) can't be found in the binary. It's because CMake thinks those paths were injected as $(CONFIGURATION) but in reality, the linker has resolved that macro.

At this point, I'm not exactly sure what is the best solution since this seems more like how to keep users from doing something potentially stupid. Ideally as a safeguard, certain compiler/linker macros that affect path expansion should fail since the resulting artifact will ultimately get corrupted. I'll leave that decision up you folks. ;)
(0034994)
Clinton Stimpson   
2014-01-25 15:00   
Thanks for narrowing that down enough. It do see how this situation comes up with doing less than ideal things such as unnecessarily including some Xcode specific variables and using discouraged CMake apis in the CMakeLists.txt.

I've attached a file as a test case. One could also use the Xcode variable $(CONFIGURATION) in place of CMAKE_CFG_INTDIR.

There are 2 problems:
1. cmake_install.cmake doesn't have the desired expansion of CMAKE_CFG_INTDIR. It already has special handling for each of the build configurations, but it might make sense to do a special expansion of CMAKE_CFG_INTDIR.

2. The Xcode project has an LD_RUNPATH_SEARCH_PATHS that contains "/path/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)" and "/path/Debug"

Brad, any thoughts on fixing this? It seems an early per-configration expansion of CMAKE_CFG_INTDIR or $(CONFIGURATION) would help avoid the problem, and give the correct results. I'm not sure how robust that will be. Perhaps issue a warning when using link_directories() with these kinds of paths?
(0034995)
Brad King   
2014-01-27 09:18   
Re 0014707:0034994: CMAKE_CFG_INTDIR is just a variable mapping to the native build system's placeholder for the configuration directory name. In Xcode's case that is $(CONFIGURATION). It is only evaluated by Xcode. When cmake_install.cmake is generated the per-config paths could be sent through an API on the global generator to expand its build-tool-specific placeholder accordingly.

As for avoiding the duplicate paths when one has placeholders and the other does not, I'm not sure what to do. Projects could use placeholders they know the native build tool supports and that CMake just trusts and passes through. I think this may have to be left documented as the responsibility of the project authors. Note that for multi-config generators, each link_directories path X actually gets used as "-L X/$(CONFIGURATION) -L X", so projects should not have to add per-config subdirectories themselves.
(0035002)
Clinton Stimpson   
2014-01-27 11:18   
Brad, which cmGlobalGenerator function can be used to expand
/path/to/${CMAKE_CFG_INTDIR} and maybe /path/to/$(CONFIGURATION)
to
/path/to/Debug (in the Debug case)
for cmake_install.cmake?

Currently the cmake_install.cmake file has
if( ... Debug ... )
  ...
   install_name_tool -delete_rpath /path/to/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)
  ...
endif()

And "cmake -P cmake_install.cmake" cannot process those Xcode variables.
(0035003)
Brad King   
2014-01-27 11:21   
Re 0014707:0035002: There isn't any method for this yet. If there were such a method, it would have to be overridden by each generator to be consistent with its GetCMakeCFGIntDir implementation.
(0035018)
Clinton Stimpson   
2014-02-01 23:25   
Fixed here:

http://cmake.org/gitweb?p=cmake.git;a=commit;h=08141a5 [^]

The Xcode project file has been fixed to avoid adding duplicate LC_RPATH entries, and the cmake_install.cmake file has been fixed to work with Xcode variables.
(0036903)
Robert Maynard   
2014-10-06 10:32   
Closing resolved issues that have not been updated in more than 4 months.