MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0013997CMakeCMakepublic2013-03-09 14:322013-10-07 10:04
ReporterKevin Burge 
Assigned ToBrad King 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformPowerPCOSAIXOS Version6.1
Product VersionCMake 2.8.10.2 
Target VersionCMake 2.8.11Fixed in VersionCMake 2.8.11 
Summary0013997: AIX shared library creation has redundant -brtl flag
DescriptionI hope this saves someone else the nightmare I have just been through trying
to figure out why I was receiving "illegal instruction" cores whenever I
tried to start my app. Note: the illegal instruction occurred before
the code ever reached main(), so, I was digging through power assembly trying
to figure out exactly what object's constructor was crashing the system before
we got to main().

After many hours of digging, and trial and error, I narrowed it down to the
link flags.

Here is how our link flags appear on the link command line:

-G -qmkshrobj -qpath=E:/home/st503/work/dev/src/mk -G -Wl,-brtl,-bnoipath -o libxdsproc.so

Note:
"-G -qmkshrobj -qpath=E:/home/st503/work/dev/src/mk" is added by us, whereas
"-G -Wl,-brtl,-bnoipath" is added by CMake.

After randomly re-ordering the flags to see if they made any difference, I discovered in the ld man page:

rtl "Enables run-time linking for the output file. This option implies
the **rtllib** and **symbolic** options." (emphasis mine)

But for the -G switch:

-G "Produces a shared object enabled for use with the run-time linker. The -G
  flag is equivalent to specifying the erok, rtl, **nortllib**, **nosymbolic**,
  noautoexp, and M:SRE options with the -b flag. Subsequent options can
  override these options." (emphasis-mine)

So, having the -brtl flag AFTER -G turns on "rtllib" and "symbolic" which -G explicitly turned off for the purpose of creating a shared library. The end result is the last option set wins.

Below is the diff of the output of "-bbindcmds" for the following two sets of
switches:

bad: -G -Wl,-brtl,-bnoipath
good: -Wl,-brtl,-bnoipath -G

diff -u bad good
--- bad 2013-03-09 12:48:05 -0600
+++ good 2013-03-09 12:48:35 -0600
@@ -1,15 +1,15 @@
 halt 4
 setopt bigtoc
-setopt noautoexp
 setopt noipath
+setopt noautoexp
 setopt rtl
-setopt rtllib
-setopt symbolic:1
+setopt nortllib <--good
+setopt symbolic:0 <--good
 setopt tmplrename
 setfflag 4
 cdtors 1 all 0 s
 savename libxdsproc.so
-filelist 75 2
+filelist 74 2
 setopt noprogram
 noentry
 i /lib/crti.o
@@ -86,8 +86,7 @@
 lib /usr/vac/lib/libxl.a
 lib /usr/vacpp/lib/libC.a
 lib /usr/lib/libc.a
-lib /usr/lib/librtl.a <--good
-exports /home/st503/tmp/xlcSEZ-ahUe
+exports /home/st503/tmp/xlcSE-5acae
 resolve
 addgl /usr/lib/glink.o
 mismatch

So, the conclusion is that for both GCC (0013352) and for XL, when building a
shared library, the linker -G switch needs to be AFTER -brtl (it's actually
redundant if -G is specified.)
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0014010closed Brad King IBM XL: Shared library object files not being compiled as position independent code 
Attached Filespatch aix.patch (1,685) 2013-03-09 14:52
https://public.kitware.com/Bug/file/4674/aix.patch
patch v2.8.10.2+0001-AIX-Do-not-use-brtl-to-create-shared-libraries-13997.patch (2,379) 2013-03-11 09:02
https://public.kitware.com/Bug/file/4676/v2.8.10.2%2B0001-AIX-Do-not-use-brtl-to-create-shared-libraries-13997.patch
Issue History
Date ModifiedUsernameFieldChange
2013-03-09 14:32Kevin BurgeNew Issue
2013-03-09 14:52Kevin BurgeFile Added: aix.patch
2013-03-10 00:34Kevin BurgeNote Added: 0032564
2013-03-11 08:47Brad KingNote Added: 0032568
2013-03-11 09:02Brad KingFile Added: v2.8.10.2+0001-AIX-Do-not-use-brtl-to-create-shared-libraries-13997.patch
2013-03-11 09:07Kevin BurgeNote Added: 0032569
2013-03-11 09:36Brad KingNote Added: 0032570
2013-03-11 09:39Brad KingNote Added: 0032571
2013-03-11 09:44Brad KingNote Added: 0032572
2013-03-11 14:39Kevin BurgeNote Added: 0032583
2013-03-11 16:00Kevin BurgeNote Edited: 0032583bug_revision_view_page.php?bugnote_id=32583#r1066
2013-03-11 16:00Kevin BurgeNote Edited: 0032583bug_revision_view_page.php?bugnote_id=32583#r1067
2013-03-12 10:30Kevin BurgeNote Added: 0032589
2013-03-12 11:57Brad KingNote Added: 0032590
2013-03-12 11:57Brad KingAssigned To => Brad King
2013-03-12 11:57Brad KingStatusnew => resolved
2013-03-12 11:57Brad KingResolutionopen => fixed
2013-03-12 11:57Brad KingFixed in Version => CMake 2.8.11
2013-03-12 11:57Brad KingTarget Version => CMake 2.8.11
2013-03-12 11:57Brad KingSummaryIncorrect linker flags causing illegal instruction core dump => AIX shared library creation has redundant -brtl flag
2013-03-13 13:10Brad KingRelationship addedrelated to 0014010
2013-10-07 10:04Robert MaynardNote Added: 0034001
2013-10-07 10:04Robert MaynardStatusresolved => closed

Notes
(0032564)
Kevin Burge   
2013-03-10 00:34   
Well, it turns out this was not the only problem, because after totally rebuilding I'm still seeing the same signal errors. (It possible that I'm not recreating the fix exactly as I had before.) It could also be that the linker on AIX is determined to cause my early death through stress.
(0032568)
Brad King   
2013-03-11 08:47   
If -G implied -brtl then I see no reason we need to have -Wl,-brtl at all when creating shared libraries. I added it 10 years ago here:

 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=9c3a6eb4 [^]

but unfortunately did not record justification in the commit message. I may have mixed up the use of the flag for linking *executables* where it is still needed. Try this patch instead:

- set(CMAKE_SHARED_LIBRARY_CREATE_${lang}_FLAGS "-G -Wl,-brtl,-bnoipath") # -shared
+ set(CMAKE_SHARED_LIBRARY_CREATE_${lang}_FLAGS "-G -Wl,-bnoipath") # -shared
(0032569)
Kevin Burge   
2013-03-11 09:07   
Hi Brad,

You are correct, it is redundant. I have the following code related to creating shared libraries on AIX, which I am still testing this today. I just upgraded the compiler (11.1) with the latest fixes to see if they will resolve the illegal instruction problem that I am still seeing. The below is what I am now testing with.

Here is how we're going around CMake's defaults:

   # The way cmake calls CreateExportList is broken for current compilers, so override cmake's xl[Cc]
   # overrides with the original values of a few things.

  set(CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS "")
  set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "")

  set(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "-brtl -Wl,-bnoipath")
  set(CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS "-brtl -Wl,-bnoipath")

   set(CMAKE_C_CREATE_SHARED_LIBRARY "<CMAKE_C_COMPILER> <CMAKE_SHARED_LIBRARY_C_FLAGS> <LANGUAGE_COMPILE_FLAGS> <LINK_FLAGS> <CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS> <CMAKE_SHARED_LIBRARY_SONAME_C_FLAG><TARGET_SONAME> -o <TARGET> <OBJECTS> <LINK_LIBRARIES>")

   set(CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_CXX_COMPILER> <CMAKE_SHARED_LIBRARY_CXX_FLAGS> <LANGUAGE_COMPILE_FLAGS> <LINK_FLAGS> <CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS> <CMAKE_SHARED_LIBRARY_SONAME_CXX_FLAG><TARGET_SONAME> -o <TARGET> <OBJECTS> <LINK_LIBRARIES>")

And, elsewhere:

set (shared_linker_flags "-qpic=large -qmkshrobj -G -Wl,-bnoipath -qpath=E:${CMAKE_SOURCE_DIR}/mk")

The effect is that we use:

"-qpic=large -qmkshrobj -G -Wl,-bnoipath" when creating shared libraries

and

"-brtl -Wl,-bnoipath" when compiling executables that link against shared libraries.

BTW, modern xlc compilers prefer using -qmkshrobj instead of using CreateExportList directly and/or -bexpall.
(0032570)
Brad King   
2013-03-11 09:36   
Use of CreateExportList was not inherited from supporting ancient tools. It was added explicitly starting in CMake 2.8.5:

 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d468a2c2 [^]
(0032571)
Brad King   
2013-03-11 09:39   
Patch to remove redundant -brtl applied:

 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=bce7a2a3 [^]
(0032572)
Brad King   
2013-03-11 09:44   
Re 0013997:0032569: What is wrong with CMake's invocation of CreateExportList?

The reason we call CreateExportList is because CMake links by passing library files explicitly e.g. /path/to/libfoo.a instead of -L/path/to -lfoo to ensure the proper libraries are used without depending on a search path. When the XL front end calls CreateExportList it incorrectly passes "libfoo.a" and ends up telling the linker to export all symbols from libfoo.a causing all of its objects to be brought in whether needed for resolving real references or not.
(0032583)
Kevin Burge   
2013-03-11 14:39   
(edited on: 2013-03-11 16:00)
Thanks Brad. Perhaps this is part of our problem.

We had to customize CreateExportList because it was not properly recognizing that variables in anonymous namespaces should not be exported (we were seeing a bunch of duplicate symbol messages on things that were in anonymous namespaces).

I did not put in the comment about the broken CreateExportList, so, I'm not sure what was considered wrong with it. If I have time, I will try removing the -qmkshrobj and make CMake use our customized CreateExportList script.

FWIW, I've started looking into statically linking all our libraries, because this is not worth it. Even with the latest fixes for the compiler related to shared library initialization, my code is still getting the "illegal instruction" signal.

Thanks again.

(Update: it looks like the person who wrote that comment resolved the problem of -qmkshrobj by changing our customized CreateExportList to ignore .so and .a files).
(0032589)
Kevin Burge   
2013-03-12 10:30   
I have a couple other things to try today, but it looks like I'm going to have to use STATIC libraries, which I tried last night and it seemed to work (got as far as main()). (FWIW, I had already taken significant efforts to remove almost all global C++ objects.) My conclusion is: I must be triggering a bug in the compiler and/or linker because even during building I received strange Duplicate Symbol and unresolved symbol errors that wouldn't normally be there on any other platform. I don't think the issue is ultimately CMake, so, I think the changes that you've made are right, they just don't fix the problem for me.

We have reverted all are shared library logic to use CMake's. I haven't tried that yet, but, I'm fairly confident I'm going to be stuck statically linking.
(0032590)
Brad King   
2013-03-12 11:57   
Thanks for reporting back. I'm resolving this issue as fixed after revising the summary to be specific to the fix that was made.
(0034001)
Robert Maynard   
2013-10-07 10:04   
Closing resolved issues that have not been updated in more than 4 months.