[CMake] Copying shared libraries in a post-build step

[Eric Wing]

On 12/9/14, Walter Gray <chrysalisx at gmail.com> wrote:
> Hey all,
> I'm working on a module that will allow me to automatically copy all the
> required .dll files as defined by well-formed import library targets to
> the appropriate location (same folder for windows, Frameworks folder for
> OSX bundle, ect).  I've got the code that scans an executable's
> INTERFACE_LINK_LIBRARIES property recursively to find all such shared
> library, however I'm running into a small problem.  I really like using
> file globbing in higher level source directories to add all appropriate
> sub-directories, but this means that sometimes a dependency will not be
> fully defined yet. This is normally fine since these things are usually
> resolved at *generation* time, but since I'm doing a manual traversal of
> the list of link libraries at config time that's not really acceptable.
> I realize I could just not do the globbing and just make sure the
> directories were setup in the correct order, but I really don't like
> making the add_subdirectory calls order dependent.
>
> One solution I've come up with is to add the targets I want to do this
> to to a global list, then iterate over that list as the last step in my
> top-level cmake lists file, but that has the issue that I can no longer
> use add_custom_command on those targets at that point.  I'm wondering 3
> things:
>
> 1)What is the reasoning behind not allowing add_custom_command on
> targets not defined in the current directory? Especially now that SOURCE
> can be modified, the restriction seems very arbitrary.
>
> 2)How stupid would it be to reserve the command using something like
> add_custom_command(TARGET ${target} POST_BUILD COMMAND
> $<TARGET_PROPERTY:COPY_SHARED_LIBS_COMMAND>)
>   then use set_property(TARGET ${target} APPEND PROPERTY
> COPY_SHARED_LIBS_COMMAND to add more copy steps to the command?
>
> 3) Am I completely missing something and there's already a totally well
> supported way of making sure that an executable's shared library
> dependencies end up in the correct directory?  I couldn't find a really
> satisfactory answer on stack overflow or the archives.
>
> Thanks!
>


I think this is a good idea if you can pull it off. I recently wrote a
lot of excruciating CMake code to do this for me. (I also was dealing
with resource and plugin code which shares similar issues.) This
actually took me a very long time to implement and it would be better
if CMake handled this automatically. Here is a list of ideas that I
needed to deal with:

- My motivation was to build a runnable product when you build. On
Visual Studio, you can't run through the debugger because none of the
.dlls are in place. Similarly on Mac, iOS, Android, and now WinRT,
there is no separation/distinction between compiling the executable
and packaging the contents. This is in my opinion of the most
problematic design issues in CMake. I also strongly feel that having
to run the install target is very wrong because it violates the normal
workflow on all these platforms and you also end up developing/testing
something different than what you ship to testers/users.

- The fix_bundle_utils didn't work for me because it did stuff in the
install stage (too late). Also, it didn't handle my usage of Mac
@rpath (not sure if that was fixed since) so I gave up on it early.


- I needed an explicit declaration mechanism to describe which
libraries needed to be bundled (or which did not). Some are system
libraries while others need to be redistributed with the app. And some
might be static libraries which you don't copy. (A .lib/.dll on Visual
Studio and a .framework on iOS is ambiguous to whether they are static
or dynamic.)

    - There was also both implicit knowledge about where they go for
each platform (had to make decisions about Linux and RPATH_ORIGIN),
and also made decisions when they were nested in subdirectories. (I
preserve the layout.)

    - For Visual Studio, that explicit mechanism had to be aware that
you link with .libs, but copy the .dll.

    - I didn't do an explicit scan of the libraries for dependencies
for multiple reasons. One important reason to me though is I want the
build to be as fast as possible (ideally on par with Xcode normally
does which is apples-to-apples in this case).

        - I don't think scanning INTERFACE_LINK_LIBRARIES is
sufficient because it won't capture what libraries each of those might
depend on. For example, I deal with a pre-build SDL_image library on
Windows which depends on libtiff, libjpeg, libpng. libpng then in turn
depends on zlib.

            - The complexity of recursively walking the dependencies,
and then also knowing which ones are system supplied libraries and
which are not is really hard, which is another reason I went the
explicit declaration route. (And shipping Linux binaries is a
nightmare; you have to make hard decisions about what is a system
library and what you need to need to ship yourself.)

- Mac/iOS framework copies were a pain because they are bundles
(directory structures) and also have symlinks, both of which CMake
does not support well for copying. I ended up using rsync on those
platforms.

- rsync had a nice advantage of resyncing if the source
framework/library changed and its a very efficient copy.

    - But rsync got me into performance overhead when I introduced
codesigning because codesigning always changes the library, causing
the library to be resynced with the original and then needing to
codesign again. (Codesigning is a slow operation.)

- I also need an explicit mechanism that signed libraries/plugins.
This signing MUST happen before Xcode tries to do the final
codesigning for your application. (I don't know if POST_BUILD is too
late or not.)

    - I was thinking that it would be nice if there was a CMake way to
declare which things need to be codesigned too. (Some kind of
SET_PROPERTIES thing.)

- In some cases, libraries get nested into subdirectories and this
information needs to be preserved for packing. Traversing the file
hierarchy relative to the bundles and preserving this information was
really nasty. I needed data structures in CMake and ended up doing
really nasty string tricks since everything is a string in CMake.


- I needed to handle things like rpath (Linux, Mac).


- Note, almost all this stuff I ended up re-applying for plugins and resources.

- I ended up dealing with icons, launch screens, application
manifests, sandboxing entitlements as completely separate entities.


There's probably a lot more.


Thanks,
Eric
-- 
Beginning iPhone Games Development
http://playcontrol.net/iphonegamebook/