[cmake-developers] Adding Swift support to CMake for Linux/Makefiles

[Eric Wing]

On 1/15/16, Brad King <brad.king at kitware.com> wrote:
> On 01/15/2016 09:47 AM, Eric Wing wrote:
>>> That is the same as for C++.  See CMAKE_PARSE_IMPLICIT_LINK_INFO and
>>
>> I looked at this file, but I still havne't groked what I need to do
>> with this yet.
>
> Somehow we need to get swift to print verbose output about the command
> line it uses to invoke the linker on the small test project CMake uses
> while enabling the language.  Then we need to parse that link line to
> get the list of libraries and link directories.  This can be a later
> step once most other things work though.
>
>> However, I'm heavily debating if 'cc' or 'clang' is the right thing to
>> do here.
>
> If the user sets LINKER_LANGUAGE to "C" and does not have main in Swift
> then CMake will want to link with the C compiler.  FYI, it is not actually
> 'cc' but the CMAKE_C_COMPILER selected, which happens to be 'cc' often.
> Anyway, we should go with CMAKE_Swift_COMPILER for now.
>
>> - First, CMake seems to be passing the flag -rdynamic to the link
>> instructions, and this is causing a "unknown argument" error. I think
>> this needs to be removed. I'm not sure where this flag is coming from.
>>
>> - Second, -Wl,-rpath,/foo/bar is also causing an "unknown argument"
>> error. As shown in the beginning of this thread, swiftc wants each
>> argument to lead with -Xlinker (and there is no -Wl,)
>> -Xlinker -rpath -Xlinker /foo/bar
>
> You'll need to add proper settings in files of the form
>
>  Modules/Platform/<os>-<id>-<lang>.cmake
>  Modules/Compiler/<id>-<lang>.cmake
>
> such as
>
>  Modules/Platform/Linux-Apple-Swift.cmake
>  Modules/Platform/Darwin-Apple-Swift.cmake
>  Modules/Compiler/Apple-Swift.cmake
>
> See Modules/Platform/Linux-NAG-Fortran.cmake for an example that
> changes from -Wl,-rpath to -Xlinker -rpath.  Similarly the
> CMAKE_SHARED_LIBRARY_LINK_<LANG>_FLAGS must not have -rdynamic
> in it.  You're getting the settings from C right now because
> your CMakeSwiftInformation.cmake file is copying them.  Instead
> leave all those out and add them as needed, preferably without
> copying from C.
>
>> Additionally, I realized I should have some other variable besides
>> <CMAKE_Swift_COMPILER>
>> Seems like I should have <CMAKE_Swift_LINKER>. I'm not sure what the
>> correct way to create this is.
>
> All the languages use CMAKE_<LANG>_COMPILER as the front-end to
> invoke for linking.  There should not need to be a separate linker
> value unless that is new to Swift semantics.
>
>> However, I currently employ a clever cheat. Since the compiler is
>> 'swift', and the linker is 'swiftc', I just do
>> '<CMAKE_Swift_COMPILER>c'
>
> So plain 'swift' cannot be used to drive the linker too?
>
>>> Note that for Ninja we actually convert our placeholders to Ninja
>>> rule placeholders and then provide the values on the actual build
>>> statement line.
>>
>> I actually don't' know anything about Ninja. I assume it is just a
>> string translation at this point since I did the other? Any guidance
>> on the mapping?
>
> Let's start with the Makefile generator.  Porting the results to
> the Ninja generator should not be too much work later.
>
>> So I need a new per-target variable in CMake for this and a way to
>> inject it into the compiler flags, maybe something like this:
>>
>> 	  "<CMAKE_Swift_COMPILER> -frontend -c  <INCLUDES> <FLAGS>
>> -import-objc-header <TARGET-Swift-BRIDGING_HEADER> -primary-file
>> <SOURCE> <Swift-SOURCES> -emit-module -module-name <TARGET> -o
>> <OBJECT>")
>>
>> With those two pieces, I think that actually makes add_executable()
>> usable in the Makefile generator.
>
> The values for the placeholders are always determined in the context
> of a specific target, so just "<Swift-BRIDGING_HEADER>" should be
> fine.
>
> Thanks,
> -Brad
>
>


This email is intended to be positive news, though there will be a lot
of details about what still needs to be done or is not quite right
yet.

So the good news is I have a basic add_executable working with Swift
on Linux via the Makefile generator.

It works with all Swift files, or intermixed C+Swift files. Bridging
header is also supported.

I took one of my small, but non-trivial programs which does a lot with
CMake to package up resources (images, audio, etc) and set rpaths and
has lots of (C) dynamic libraries it links to which also get shipped
with the bundle to make a self-contained app (rpath), and ported the
core parts to Swift. CMake handled this pretty much transparently (had
to enable_language(Swift), set the bridging header, and there are
Swift Standard Library things to sort out later). So overall, I’m
really happy with this.



So now the details.

- I’m still copying over some CFLAGS to Swift_FLAGS. For example, I
couldn’t figure out what was providing the ‘-I’ flag for include
paths. Rules like this still need to be rewritten for Swift.

- TODO: I need to look at Swift's DEFINE system.

- Link flags for -rpath are now rewritten (and -rdynamic is removed)

- Added  set_target_property(my_exe SWIFT_BRIDGING_HEADER
“/foo/bar/mybridge.h”). There is a token in the .cmake description for
<Swift-BRIDGING_HEADER>, but the full flag for -import-objc-header
/foo/bar/mybridge.h is generated in the C++ side. This is because I
need to dynamically decide whether this flag gets submitted or not
based on whether a header was provided. (I can’t have a
-import-objc-header without an argument.)

- TODO: Unify SWIFT_BRIDGING_HEADER with the Xcode generator. Right
now, we rely on the user setting the Xcode property directly which has
a different name. Seems like if they set SWIFT_BRIDGING_HEADER, it
should set the Xcode one for them.

- TODO: Need to do Platform/Darwin-Apple-Swift.cmake.

- Note: The new file is Linux-unknown-Swift.cmake. When I compile
Swift myself, there is no company or organization identifier (swift
--version) unlike the one Apple ships with Xcode, and CMake refers to
the identifier as ‘unknown’. I made the file reflect this, but maybe
we should rename this later (maybe Swift_org).


- Quirks/Bugs/Limitations about add_executable and the main symbol:

    I learned a few details and there are some quirks.
    Swift treats main.swift specially and will generate a main symbol.
    If you have a C main function, there is an edge case bug...
    If you have only 1 Swift file (and your C main), Swift will
autogenerate a main symbol for Swift. This will cause a duplicate
symbol problem and fail to link.
    If you have multiple Swift files (and your C main), this will link
correctly, with the exception that you must not have a main.swift
file.

I still need to look deeper into how Xcode deals with this, but a
quick glance, and I noticed an extra build stage between the object
files and the linking. I think this extra stage in Xcode takes all the
Swift object files and makes a .swiftmodule (library?) out of them and
then may submit this to the linker. This could be the mechanism Xcode
to avoid duplicate symbol problems, though as I said, I haven’t
dissected this for real yet. But I’m thinking this may automatically
come into more focus as we work on the library/module support for
Swift.


- To answer your question about whether ‘swift’ instead of ’swiftc’
can be used for linking, I think the answer is yes, but I have been
unable to figure out the correct incantation. I usually get a link
failure and a bunch of console spew which contains non-ASCII
characters and my terminal starts beeping and going crazy. It’s
unpleasant to debug. And I can’t find one real world example of using
swift for just linking. It’s either clang, ld, or swiftc. I’m also
worried about how to determine the SDK paths when I’m not using
swiftc, since I don’t provide that information explicitly when on Mac.
Interesting Trivia: The swiftc linker calls clang++ under the hood for
linking.


- Swift Standard (or Core) Libraries: Swift comes with multiple
standard libraries. There is swiftCore which I think contains a lot of
the fundamental types like Int and Double and containers like Array
and Dictionary. (It’s still fuzzy to me where the line between
language and library begins.)
https://developer.apple.com/library/ios/documentation/Swift/Reference/Swift_Int_Structure/index.html#//apple_ref/swift/struct/s:Si

Then there is Foundation which is a Swift rewrite of Apple’s Cocoa
(Obj-C) Foundation, which contains a large set of classes and
convenience methods for non-GUI stuff, like NSString, NSURL,
NSURLSession, etc. It is mostly incompletely right now on Linux.

Then they also plan to include libdispatch and XCTest.
https://swift.org/core-libraries/


Additionally, they provide a wrapper interface to the C standard
library on the platform. This library has either the darwin or glibc
name in it depending on situation. So I think something as simple as
M_PI from C’s math.h will bring in the C library dependency.


As far as I can tell, the swiftc linker knows how to automatically
link to these libraries as needed depending if your code uses them or
not. So the nice thing is that the CMake user scripts don’t currently
need you to specify anything explicitly for these libraries. However,
the one complication for me is that I am trying to ship stand-alone,
redistributable application binaries, which means I need to copy these
libraries with my application on non-Apple platforms. (I’ve already
been setting path $ORIGIN on Linux). So I think CMake should provide
some variables to the full paths to these libraries since users like
me will want to refer to them. The question is whether these are set
as part of the language bootstrap scripts, or a separate
FindSwiftCoreLibraries.cmake module the user needs to manually
include. (And unfortunately, for me, I don’t see a good way to know
which libraries I actually need to copy without running ldd on the
binary unless I know a priori. I know for now, I’m avoiding Foundation
since it is unfinished and probably going to be huge binary and I
don’t need it for my stuff.)


Thanks,
Eric