[cmake-developers] [CMake 0013997]: Incorrect linker flags causing illegal instruction core dump

[Mantis Bug Tracker]

The following issue has been SUBMITTED. 
====================================================================== 
http://www.cmake.org/Bug/view.php?id=13997 
====================================================================== 
Reported By:                Kevin Burge
Assigned To:                
====================================================================== 
Project:                    CMake
Issue ID:                   13997
Category:                   CMake
Reproducibility:            always
Severity:                   major
Priority:                   normal
Status:                     new
====================================================================== 
Date Submitted:             2013-03-09 14:32 EST
Last Modified:              2013-03-09 14:32 EST
====================================================================== 
Summary:                    Incorrect linker flags causing illegal instruction
core dump
Description: 
I hope this saves someone else the nightmare I have just been through trying
to figure out why I was receiving "illegal instruction" cores whenever I
tried to start my app.  Note: the illegal instruction occurred before
the code ever reached main(), so, I was digging through power assembly trying
to figure out exactly what object's constructor was crashing the system before
we got to main().

After many hours of digging, and trial and error, I narrowed it down to the
link flags.

Here is how our link flags appear on the link command line:

-G -qmkshrobj -qpath=E:/home/st503/work/dev/src/mk   -G -Wl,-brtl,-bnoipath  -o
libxdsproc.so

Note:
"-G -qmkshrobj -qpath=E:/home/st503/work/dev/src/mk" is added by us, whereas
"-G -Wl,-brtl,-bnoipath" is added by CMake.

After randomly re-ordering the flags to see if they made any difference, I
discovered in the ld man page:

rtl "Enables run-time linking for the output file. This option implies
the **rtllib** and **symbolic** options." (emphasis mine)

But for the -G switch:

-G "Produces a shared object enabled for use with the run-time linker. The -G
  flag is equivalent to specifying the erok, rtl, **nortllib**, **nosymbolic**,
  noautoexp, and M:SRE options with the -b flag. Subsequent options can
  override these options." (emphasis-mine)

So, having the -brtl flag AFTER -G turns on "rtllib" and "symbolic" which -G
explicitly turned off for the purpose of creating a shared library.  The end
result is the last option set wins.

Below is the diff of the output of "-bbindcmds" for the following two sets of
switches:

bad: -G -Wl,-brtl,-bnoipath
good: -Wl,-brtl,-bnoipath -G 

diff -u bad good
--- bad 2013-03-09 12:48:05 -0600
+++ good        2013-03-09 12:48:35 -0600
@@ -1,15 +1,15 @@
 halt 4
 setopt bigtoc
-setopt noautoexp
 setopt noipath
+setopt noautoexp
 setopt rtl
-setopt rtllib
-setopt symbolic:1
+setopt nortllib                   <--good
+setopt symbolic:0                 <--good
 setopt tmplrename
 setfflag 4
 cdtors 1 all 0 s
 savename libxdsproc.so
-filelist 75 2
+filelist 74 2
 setopt noprogram
 noentry
 i /lib/crti.o
@@ -86,8 +86,7 @@
 lib /usr/vac/lib/libxl.a
 lib /usr/vacpp/lib/libC.a
 lib /usr/lib/libc.a
-lib /usr/lib/librtl.a              <--good
-exports /home/st503/tmp/xlcSEZ-ahUe
+exports /home/st503/tmp/xlcSE-5acae
 resolve
 addgl /usr/lib/glink.o
 mismatch

So, the conclusion is that for both GCC
(http://www.cmake.org/Bug/view.php?id=13352) and for XL, when building a
shared library, the linker -G switch needs to be AFTER -brtl (it's actually
redundant if -G is specified.)

====================================================================== 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2013-03-09 14:32 Kevin Burge    New Issue                                    
======================================================================