[CMake] NVCC and Shared Library Full Path Linking

[James Bigler]

The way I implemented separable compilation it doesn't use nvcc to link.
 It relies on the host compiler to do the linking.  I'm not sure at first
blush why you are having this problem.  One thing that should probably be
done is to change this line of code in FindCUDA.cmake:

    # For now we are ignoring all the configuration specific flags.
    set(nvcc_flags)
    CUDA_PARSE_NVCC_OPTIONS(nvcc_flags ${options})


To this to pass the CUDA_NVCC_FLAGS in.

    # For now we are ignoring all the configuration specific flags.
    set(nvcc_flags ${CUDA_NVCC_FLAGS})
    CUDA_PARSE_NVCC_OPTIONS(nvcc_flags ${options})


On Mon, Dec 16, 2013 at 2:25 PM, MARTIN, ROBERT S CTR USAF AFMC AFRL/RQRS <
robert.martin.81.ctr at us.af.mil> wrote:

> CMake Group-
>
> I'm trying to build a CUDA project with NVCC.  The project uses 'separate
> compilation'.  Unfortunately, 'separate compilation' breaks in a variety of
> ways when I try to use the version in FindCUDA.cmake.  First, it doesn't
> include CUDA_NVCC_FLAGS in the -dlink phase so it immediately complains
> that
> the dlink flag can't be used without sm_20+.  That has already been noted
> as
> a bug (http://public.kitware.com/Bug/view.php?id=14238) and there seems to
> be some discussion about it being inappropriate to default to some sm_
> version.  Since multiple sm_ options can be listed but only one arch flag,
> it seems like this may need a more advanced way to be dealt with.  Anyway,
> after modifying things to add a -arch=sm_20 to the command line, it works
> for a while but then cannot dlink one of my objects because of:
> 'nvlink error :  Undefined reference to '_Z3minRK6float3S1_' in '(..some
> path...)_generated_(...some file...).cu.o'
>
>  I can't find this problem because I don't really understand function name
> mangling, but it looks like it cannot link to a 'min' function probably.
> It's not one of the "__cudaRegisterLinkedBinary..." type issues though, and
> so I'm not clear why nvlink is trying to resolve all the symbols for an
> intermediate object anyway.
>
> As a workaround for all this, I tried adding '-dc' to the normal compiles
> and linking with nvcc instead of gcc so that nvcc can do device and host
> linking in one pass.
> (
> http://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#using-sepa
> rate-compilation-in-cuda ... "nvcc <objects>").  This almost works except
> for a few link command line incompatibilities.  First, nvcc doesn't know
> what to do with '-Wl,-rpath...'.  It needs some '-Xlinker' in front of that
> to know where to send it.  I can get around that by just turning off rpath
> with 'set(CMAKE_SKIP_RPATH TRUE)', but it's not a great solution.    I also
> have to remove the '-rdynamic' flag by setting
> CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS to nothing (""). The last issue is that
> for some reason on some machines CMake tries to use full paths for shared
> libraries rather than -l & -L and nvcc chokes on the extension (static
> libraries with .a are fine).   The error message is:
>
> Nvlink fatal   : Unsupported file type '/usr/lib/openmpi/lib/libmpi_cxx.so'
>
> If I replace the '/usr/lib/openmpi/lib/libmpi_cxx.so' with
> '-L/usr/lib/openmpi/lib/ -lmpi_cxx', it works.  When I look at the ldd
> output, it's even correctly still linking to the .so (rather than a static
> library).  Interestingly enough, on a different machine with a similar
> setup
> the FindMPI module adds the '-L -l' options instead of the full path (mpich
> instead of openmpi though?).  I found this page on the cmake wiki that
> suggests getting rid of '-L/-l' as an 'improvement':
>
> http://www.cmake.org/Wiki/CMake:Improving_Find*_Modules#Converting_-L.2F-l_f
> lags
>
> Is this currently an encouraged approach?  Is there a way to control the
> behavior in cmake to force CMake to choose using '-L/-l' instead of full
> paths?  I realize this could also be considered more of a bug with nvlink
> than with cmake, but I figured I could try to start here.  It would be a
> simple enough regular expression to break the full path into the two
> pieces,
> but I'm pretty new to CMake and I'm not sure how this could be
> accomplished.
> If I could just get control of this behavior, I'd have something that would
> work for now until FindCUDA can get sorted out a little more with respect
> to
> all the CUDA 5+ changes and/or more tightly integrated into CMake like
> other
> compilers tend to be.
>
> Thanks for any help people can provide.
>
>
>
>
>
> Robert Martin, ERC Inc.
> EP Modeling and Simulation Group
> In-Space Propulsion Branch
> Air Force Research Laboratory
> 661-275-6659
>
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20131223/8beb7143/attachment.htm>