View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0013997CMakeCMakepublic2013-03-09 14:322013-10-07 10:04
ReporterKevin Burge 
Assigned ToBrad King 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformPowerPCOSAIXOS Version6.1
Product VersionCMake 2.8.10.2 
Target VersionCMake 2.8.11Fixed in VersionCMake 2.8.11 
Summary0013997: AIX shared library creation has redundant -brtl flag
DescriptionI hope this saves someone else the nightmare I have just been through trying
to figure out why I was receiving "illegal instruction" cores whenever I
tried to start my app. Note: the illegal instruction occurred before
the code ever reached main(), so, I was digging through power assembly trying
to figure out exactly what object's constructor was crashing the system before
we got to main().

After many hours of digging, and trial and error, I narrowed it down to the
link flags.

Here is how our link flags appear on the link command line:

-G -qmkshrobj -qpath=E:/home/st503/work/dev/src/mk -G -Wl,-brtl,-bnoipath -o libxdsproc.so

Note:
"-G -qmkshrobj -qpath=E:/home/st503/work/dev/src/mk" is added by us, whereas
"-G -Wl,-brtl,-bnoipath" is added by CMake.

After randomly re-ordering the flags to see if they made any difference, I discovered in the ld man page:

rtl "Enables run-time linking for the output file. This option implies
the **rtllib** and **symbolic** options." (emphasis mine)

But for the -G switch:

-G "Produces a shared object enabled for use with the run-time linker. The -G
  flag is equivalent to specifying the erok, rtl, **nortllib**, **nosymbolic**,
  noautoexp, and M:SRE options with the -b flag. Subsequent options can
  override these options." (emphasis-mine)

So, having the -brtl flag AFTER -G turns on "rtllib" and "symbolic" which -G explicitly turned off for the purpose of creating a shared library. The end result is the last option set wins.

Below is the diff of the output of "-bbindcmds" for the following two sets of
switches:

bad: -G -Wl,-brtl,-bnoipath
good: -Wl,-brtl,-bnoipath -G

diff -u bad good
--- bad 2013-03-09 12:48:05 -0600
+++ good 2013-03-09 12:48:35 -0600
@@ -1,15 +1,15 @@
 halt 4
 setopt bigtoc
-setopt noautoexp
 setopt noipath
+setopt noautoexp
 setopt rtl
-setopt rtllib
-setopt symbolic:1
+setopt nortllib <--good
+setopt symbolic:0 <--good
 setopt tmplrename
 setfflag 4
 cdtors 1 all 0 s
 savename libxdsproc.so
-filelist 75 2
+filelist 74 2
 setopt noprogram
 noentry
 i /lib/crti.o
@@ -86,8 +86,7 @@
 lib /usr/vac/lib/libxl.a
 lib /usr/vacpp/lib/libC.a
 lib /usr/lib/libc.a
-lib /usr/lib/librtl.a <--good
-exports /home/st503/tmp/xlcSEZ-ahUe
+exports /home/st503/tmp/xlcSE-5acae
 resolve
 addgl /usr/lib/glink.o
 mismatch

So, the conclusion is that for both GCC (0013352) and for XL, when building a
shared library, the linker -G switch needs to be AFTER -brtl (it's actually
redundant if -G is specified.)
TagsNo tags attached.
Attached Filespatch file icon aix.patch [^] (1,685 bytes) 2013-03-09 14:52 [Show Content]
patch file icon v2.8.10.2+0001-AIX-Do-not-use-brtl-to-create-shared-libraries-13997.patch [^] (2,379 bytes) 2013-03-11 09:02 [Show Content]

 Relationships
related to 0014010closedBrad King IBM XL: Shared library object files not being compiled as position independent code 

  Notes
(0032564)
Kevin Burge (reporter)
2013-03-10 00:34

 Well, it turns out this was not the only problem, because after totally rebuilding I'm still seeing the same signal errors. (It possible that I'm not recreating the fix exactly as I had before.) It could also be that the linker on AIX is determined to cause my early death through stress.
(0032568)
Brad King (manager)
2013-03-11 08:47

 If -G implied -brtl then I see no reason we need to have -Wl,-brtl at all when creating shared libraries. I added it 10 years ago here:

 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=9c3a6eb4 [^]

but unfortunately did not record justification in the commit message. I may have mixed up the use of the flag for linking *executables* where it is still needed. Try this patch instead:

- set(CMAKE_SHARED_LIBRARY_CREATE_${lang}_FLAGS "-G -Wl,-brtl,-bnoipath") # -shared
+ set(CMAKE_SHARED_LIBRARY_CREATE_${lang}_FLAGS "-G -Wl,-bnoipath") # -shared
(0032569)
Kevin Burge (reporter)
2013-03-11 09:07

 Hi Brad,

You are correct, it is redundant. I have the following code related to creating shared libraries on AIX, which I am still testing this today. I just upgraded the compiler (11.1) with the latest fixes to see if they will resolve the illegal instruction problem that I am still seeing. The below is what I am now testing with.

Here is how we're going around CMake's defaults:

   # The way cmake calls CreateExportList is broken for current compilers, so override cmake's xl[Cc]
   # overrides with the original values of a few things.

  set(CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS "")
  set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "")

  set(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "-brtl -Wl,-bnoipath")
  set(CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS "-brtl -Wl,-bnoipath")

   set(CMAKE_C_CREATE_SHARED_LIBRARY "<CMAKE_C_COMPILER> <CMAKE_SHARED_LIBRARY_C_FLAGS> <LANGUAGE_COMPILE_FLAGS> <LINK_FLAGS> <CMAKE_SHARED_LIBRARY_CREATE_C_FLAGS> <CMAKE_SHARED_LIBRARY_SONAME_C_FLAG><TARGET_SONAME> -o <TARGET> <OBJECTS> <LINK_LIBRARIES>")

   set(CMAKE_CXX_CREATE_SHARED_LIBRARY "<CMAKE_CXX_COMPILER> <CMAKE_SHARED_LIBRARY_CXX_FLAGS> <LANGUAGE_COMPILE_FLAGS> <LINK_FLAGS> <CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS> <CMAKE_SHARED_LIBRARY_SONAME_CXX_FLAG><TARGET_SONAME> -o <TARGET> <OBJECTS> <LINK_LIBRARIES>")

And, elsewhere:

set (shared_linker_flags "-qpic=large -qmkshrobj -G -Wl,-bnoipath -qpath=E:${CMAKE_SOURCE_DIR}/mk")

The effect is that we use:

"-qpic=large -qmkshrobj -G -Wl,-bnoipath" when creating shared libraries

and

"-brtl -Wl,-bnoipath" when compiling executables that link against shared libraries.

BTW, modern xlc compilers prefer using -qmkshrobj instead of using CreateExportList directly and/or -bexpall.
(0032570)
Brad King (manager)
2013-03-11 09:36

 Use of CreateExportList was not inherited from supporting ancient tools. It was added explicitly starting in CMake 2.8.5:

 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d468a2c2 [^]
(0032571)
Brad King (manager)
2013-03-11 09:39

 Patch to remove redundant -brtl applied:

 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=bce7a2a3 [^]
(0032572)
Brad King (manager)
2013-03-11 09:44

 Re 0013997:0032569: What is wrong with CMake's invocation of CreateExportList?

The reason we call CreateExportList is because CMake links by passing library files explicitly e.g. /path/to/libfoo.a instead of -L/path/to -lfoo to ensure the proper libraries are used without depending on a search path. When the XL front end calls CreateExportList it incorrectly passes "libfoo.a" and ends up telling the linker to export all symbols from libfoo.a causing all of its objects to be brought in whether needed for resolving real references or not.
(0032583)
Kevin Burge (reporter)
2013-03-11 14:39
edited on: 2013-03-11 16:00

 Thanks Brad. Perhaps this is part of our problem.

We had to customize CreateExportList because it was not properly recognizing that variables in anonymous namespaces should not be exported (we were seeing a bunch of duplicate symbol messages on things that were in anonymous namespaces).

I did not put in the comment about the broken CreateExportList, so, I'm not sure what was considered wrong with it. If I have time, I will try removing the -qmkshrobj and make CMake use our customized CreateExportList script.

FWIW, I've started looking into statically linking all our libraries, because this is not worth it. Even with the latest fixes for the compiler related to shared library initialization, my code is still getting the "illegal instruction" signal.

Thanks again.

(Update: it looks like the person who wrote that comment resolved the problem of -qmkshrobj by changing our customized CreateExportList to ignore .so and .a files).
(0032589)
Kevin Burge (reporter)
2013-03-12 10:30

 I have a couple other things to try today, but it looks like I'm going to have to use STATIC libraries, which I tried last night and it seemed to work (got as far as main()). (FWIW, I had already taken significant efforts to remove almost all global C++ objects.) My conclusion is: I must be triggering a bug in the compiler and/or linker because even during building I received strange Duplicate Symbol and unresolved symbol errors that wouldn't normally be there on any other platform. I don't think the issue is ultimately CMake, so, I think the changes that you've made are right, they just don't fix the problem for me.

We have reverted all are shared library logic to use CMake's. I haven't tried that yet, but, I'm fairly confident I'm going to be stuck statically linking.
(0032590)
Brad King (manager)
2013-03-12 11:57

 Thanks for reporting back. I'm resolving this issue as fixed after revising the summary to be specific to the fix that was made.
(0034001)
Robert Maynard (manager)
2013-10-07 10:04

 Closing resolved issues that have not been updated in more than 4 months.

 Issue History
Date Modified Username Field Change
2013-03-09 14:32 Kevin Burge New Issue
2013-03-09 14:52 Kevin Burge File Added: aix.patch
2013-03-10 00:34 Kevin Burge Note Added: 0032564
2013-03-11 08:47 Brad King Note Added: 0032568
2013-03-11 09:02 Brad King File Added: v2.8.10.2+0001-AIX-Do-not-use-brtl-to-create-shared-libraries-13997.patch
2013-03-11 09:07 Kevin Burge Note Added: 0032569
2013-03-11 09:36 Brad King Note Added: 0032570
2013-03-11 09:39 Brad King Note Added: 0032571
2013-03-11 09:44 Brad King Note Added: 0032572
2013-03-11 14:39 Kevin Burge Note Added: 0032583
2013-03-11 16:00 Kevin Burge Note Edited: 0032583
2013-03-11 16:00 Kevin Burge Note Edited: 0032583
2013-03-12 10:30 Kevin Burge Note Added: 0032589
2013-03-12 11:57 Brad King Note Added: 0032590
2013-03-12 11:57 Brad King Assigned To => Brad King
2013-03-12 11:57 Brad King Status new => resolved
2013-03-12 11:57 Brad King Resolution open => fixed
2013-03-12 11:57 Brad King Fixed in Version => CMake 2.8.11
2013-03-12 11:57 Brad King Target Version => CMake 2.8.11
2013-03-12 11:57 Brad King Summary Incorrect linker flags causing illegal instruction core dump => AIX shared library creation has redundant -brtl flag
2013-03-13 13:10 Brad King Relationship added related to 0014010
2013-10-07 10:04 Robert Maynard Note Added: 0034001
2013-10-07 10:04 Robert Maynard Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team