[CMake] library path stripping
Michael Hertling
mhertling at online.de
Sat Jun 11 10:21:24 EDT 2011
On 06/10/2011 12:57 PM, Andreas Naumann wrote:
> Am 10.06.2011 00:16, schrieb Michael Hertling:
>> On 06/09/2011 08:18 AM, Andreas Naumann wrote:
>>
>>> Am 09.06.2011 07:35, schrieb Michael Hertling:
>>>
>>>> On 06/09/2011 07:13 AM, Andreas Naumann wrote:
>>>>
>>>>
>>>>> Am 08.06.2011 20:43, schrieb Andreas Pakulat:
>>>>>
>>>>>
>>>>>> On 08.06.11 20:00:54, Andreas Naumann wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Am 08.06.2011 15:02, schrieb Eric Noulard:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> 2011/6/8 Andreas Naumann<Andreas-Naumann at gmx.net>:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Am 08.06.2011 11:56, schrieb Eric Noulard:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> 2011/6/8 Andreas Naumann<Andreas-Naumann at gmx.net>:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi @all,
>>>>>>>>>>>
>>>>>>>>>>> I have some problem with the library usage in cmake.
>>>>>>>>>>>
>>>>>>>>>>> It seems to me, that cmake removes the full path of the library, if the
>>>>>>>>>>> path
>>>>>>>>>>> is in the environment variable LIBRARY_PATH.
>>>>>>>>>>> This behaviour cause problems at our system, such that the linker links
>>>>>>>>>>> against the wrong library.
>>>>>>>>>>>
>>>>>>>>>>> Is there an option to avoid this splitting?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Did you read that:
>>>>>>>>>> http://www.cmake.org/Wiki/CMake_RPATH_handling
>>>>>>>>>>
>>>>>>>>>> Which version of CMake are you using?
>>>>>>>>>> On which system ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> We are using version 2.8.2 and 2.8.4 on Debian and Suse Linux.
>>>>>>>>>
>>>>>>>>> I've read the hints on RPATH handling, but there it is said, that:
>>>>>>>>> "By default if you don't change any RPATH related settings, CMake will link
>>>>>>>>> the executables and shared libraries with full RPATH to all used libraries
>>>>>>>>> in the build tree"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Yes.
>>>>>>>> And you did not mention it but the probleme you have occurs when using
>>>>>>>> the executable **directly from the build dir** right?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> what do you mean with "using"? I cannot even link the executable,
>>>>>>> because cmake removes the path from the library without adding the
>>>>>>> directory to the library directories.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> This means, if I don't set anything related to RPATH, cmake should not strip
>>>>>>>>> the path from the library, should it?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> No it shoudn't for the binary in the buitd tree but...
>>>>>>>> if you do "make install" the installed binaries will have no RPATH unless
>>>>>>>> you set
>>>>>>>>
>>>>>>>> SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
>>>>>>>>
>>>>>>>> # add the automatically determined parts of the RPATH
>>>>>>>> # which point to directories outside the build tree to the install RPATH
>>>>>>>> SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> I don't even want to install, just build and use.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>> I played with the example and it was a bit hard to understand, when the path
>>>>>>>>> is stripped from the library.
>>>>>>>>> The problem arises before linking the executable.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> This is a different issue.
>>>>>>>> Could you copy/paste the
>>>>>>>>
>>>>>>>> add_executable
>>>>>>>> and
>>>>>>>> target_link_libraries
>>>>>>>>
>>>>>>>> statement you use for the offending executable?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> simple:
>>>>>>> project("test")
>>>>>>>
>>>>>>> cmake_minimum_required(VERSION 2.8)
>>>>>>> set(MYLIB /home/andreas/cmake_test/lib/libfoo.so)
>>>>>>> add_executable(foo_exec test.cc)
>>>>>>> target_link_libraries(foo_exec ${MYLIB})
>>>>>>>
>>>>>>> So the executable gets the full name and the example works, if the
>>>>>>> environment variable library_path is not set to
>>>>>>> /home/andreas/cmake_test/lib. The link command is:
>>>>>>>
>>>>>>> /usr/bin/c++ CMakeFiles/foo_exec.dir/test.cc.o -o foo_exec
>>>>>>> -rdynamic /home/andreas/cmake_test/lib/libfoo.so
>>>>>>> -Wl,-rpath,/home/andreas/cmake_test/lib
>>>>>>>
>>>>>>> If I set LIBRARY_PATH to /home/andreas/cmake_test/lib, the directory
>>>>>>> is stripped and the link-command gets:
>>>>>>>
>>>>>>> /usr/bin/c++ CMakeFiles/foo_exec.dir/test.cc.o -o foo_exec
>>>>>>> -rdynamic -lfoo
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I can reproduce this here with CMake 2.8.4.
>>>>>>
>>>>>> And before someone else asks (since I thought that might be the error),
>>>>>> the envvar is really "LIBRARY_PATH", i.e. not LD_LIBRARY_PATH.
>>>>>>
>>>>>> The strange thing is that there's no occurrence of this specific string
>>>>>> anywhere in the cmake sources. There are all kinds of occurrence of
>>>>>> LD_... or CMAKE_... but no string matching just LIBRARY_PATH. So its
>>>>>> unclear why CMake respects that envvar.
>>>>>>
>>>>>> Andreas
>>>>>>
>>>>>> _______________________________________________
>>>>>> Powered by www.kitware.com
>>>>>>
>>>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>>>
>>>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>>>
>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>> http://www.cmake.org/mailman/listinfo/cmake
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> The problem is related to the compiler, not to the linker. The
>>>>> gcc-documentation
>>>>> http://gcc.gnu.org/onlinedocs/gcc-4.6.0/gcc/Environment-Variables.html#Environment-Variables
>>>>> says:
>>>>>
>>>>> LIBRARY_PATH
>>>>> The value of LIBRARY_PATH is a colon-separated list of directories, much
>>>>> like PATH. When configured as a native compiler, GCC tries the
>>>>> directories thus specified when searching for special linker files, if
>>>>> it can't find them using GCC_EXEC_PREFIX. Linking using GCC also uses
>>>>> these directories when searching for ordinary libraries for the -l
>>>>> option (but directories specified with -L come first).
>>>>>
>>>>> CMake seems to ask the compiler for implicitly used link directories and
>>>>> removes each path in this list from the libraries in the directories.
>>>>> Because this information comes from the compiler, cmake cannot
>>>>> distinguish between working and not working directories.
>>>>> My question is now, why does cmake remove the paths? If the use
>>>>> specifies a library with full path, he wants EXACTLY this library.
>>>>>
>>>>> Currently, we print a warning, if the variable LIBRARY_PATH is set in
>>>>> the environment.
>>>>>
>>>>> Andreas
>>>>>
>>>>>
>>>> During the initial examination of the compiler ("Detecting CXX compiler
>>>> ABI info"), the test program is linked with -v, the output captured and
>>>> the link line scanned for -L arguments, thus detecting the directories
>>>> in the LIBRARY_PATH. The results go into the CMAKE_CXX_IMPLICIT_LINK_
>>>> DIRECTORIES variable; use --debug-trycompile and look into CMakeFiles/
>>>> CMakeOutput.log in order to see. Libraries in a directory mentioned in
>>>> this variable are treated as system libraries and will be linked with
>>>> -l instead of a full path, see [1] for the reasons. If you need the
>>>> library's full path despite of this mechanism, use imported targets.
>>>>
>>>> Regards,
>>>>
>>>> Michael
>>>>
>>>>
>>>>
>>>>> [1] http://www.cmake.org/Wiki/CMake_2.6_Notes#Linking_to_System_Libraries
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> http://www.cmake.org/mailman/listinfo/cmake
>>>>
>>>>
>>>>
>>> @Michael:
>>>
>>> Thank you very much for this hint. It helps to understand and track down
>>> the problem.
>>> In our special case, it is a bad combination of environment variable
>>> setting and compiler configuration.
>>>
>>> The old libraries, we are looking for are in /usr/lib[64], which IS a
>>> standard system path. The new libraries are on /some/other/path AND the
>>> LIBRARY_PATH variable is set.
>>> Now, it depends on the compiler configuration, in which order the
>>> LIBRARY_PATH and other SYSTEM_PATHS are combined, and the paths in
>>> $LIBRARY_PATH are after the other system paths..
>>>
>>> But this is a part of the compiler which cmake has no access to.
>>>
>> IIRC, trouble of this kind has been the reason for CMake to prefer full
>> paths for libraries, and if you get full-path-specified libraries into
>> the link command - either immediately or by imported targets - it does
>> not matter which directories in which order the compiler searches. So,
>> I'm not sure if I understand why your problem still persists.
>>
> The problem persists, because the compiler tells cmake, the directory in
> LIBRARY_PATH would be a system directory. That's why, the library is
> linked with -l. Now, it is up to the compiler configuration to set the
> directories in LIBRARY_PATH before the other system directories. At our
> system, this is not the case.
And that's the reason why you must treat the concerned libraries as
imported targets if you want to ensure that they are referred to by
full paths. Otherwise, you are in danger that the compiler will use
-l to link binaries against them without having a chance to prevent
it, and the order of system directories will matter in such a case.
Alternatively, you might perhaps remove the LIBRARY_PATH from the
CMAKE_<LANG>_IMPLICIT_LINK_DIRECTORIES variables.
>>> Working with imported targets gets complicated, because the libraries
>>> with full path are given by the user and adding each one as imported
>>> target looks a bit odd.
>>>
>> If I'm not mistaken, referring to external libraries, i.e. libraries
>> not built by the project, is quite exactly what imported targets are
>> meant for; thus, their usage should not be odd but normal. If their
>> additional effort with ADD_LIBRARY(... IMPORTED) and the IMPORTED_
>> LOCATION properties is annoying, you can write a function for this
>> purpose, or are there other reasons why using imported targets for
>> user-supplied libraries gets complicated?
>>
>>
> This means, I should call
>
> ADD_LIBRARY(libname IMPORTED)
>
> for each library I look for?
Yes.
> And than also set IMPORTED_LOCATION?
Yes. Usually, that's slightly more work than referring to the affected
libraries immediately, but it can be simplified with a macro/function
and, among other things, does provide much more flexibility when the
libraries to link against are configuration-specific.
> CMake should prefer fullpaths of libraries, at least for libraries in
> non-system directories, shouldn't it?
CMake *does* do so, but which directories are considered as system
ones and which are not? As you've said yourself, it's the compiler
that tells CMake about these directories, and the reason why this
full-path approach is dropped for libraries therein is explained
in the wiki I provided the link to.
> When should I use add_library( ... IMPORTED) ?
Basically, IMO, one should use this for each external library unless
there's a reason not to do so, typically with architecture/hardware-
specific libraries the compiler probably knows better where to find
them. But again: Even then, you can force to link against full-path
specified libraries by using imported targets.
Regards,
Michael
More information about the CMake
mailing list