View Issue Details [ Jump to Notes ] | [ Print ] | ||||||||
ID | Project | Category | View Status | Date Submitted | Last Update | ||||
0012284 | CMake | CPack | public | 2011-06-17 16:57 | 2012-03-06 08:37 | ||||
Reporter | Eric NOULARD | ||||||||
Assigned To | David Cole | ||||||||
Priority | normal | Severity | minor | Reproducibility | always | ||||
Status | closed | Resolution | fixed | ||||||
Platform | OS | Linux | OS Version | ||||||
Product Version | CMake 2.8.4 | ||||||||
Target Version | CMake 2.8.6 | Fixed in Version | CMake 2.8.6 | ||||||
Summary | 0012284: User Install time created symlinks are not preserved in TGZ | ||||||||
Description | Reporte by John Cary on the ML http://www.cmake.org/pipermail/cmake/2011-June/044940.html [^] | ||||||||
Steps To Reproduce | Attached is a minimal standalone example that illustrates the problem. If build dir is next to the directory resulting from untarring this, do cmake -DCMAKE_INSTALL_PREFIX=$HOME/tmp/cmtprefix ../cmt make all install # You will see that in the installation, the top cmtdir is a link: numbersix.cary$ ls -l $HOME/tmp/cmtprefix/cmtdir lrwxrwxr-x 1 cary research 10 Jun 17 11:16 /Users/cary/tmp/cmtprefix/cmtdir -> bin/cmtdir # then do make package tar xzf cmt-1.1.1-Darwin.tar.gz # and you will see that the top-level cmtdir is a directory: numbersix.cary$ ls -l cmt-1.1.1-Darwin total 0 drwxrwxr-x 3 cary research 102 Jun 17 11:25 bin drwxrwxr-x 3 cary research 102 Jun 17 11:25 cmtdir such that two copies of the binary are installed | ||||||||
Tags | No tags attached. | ||||||||
Attached Files | ![]() ![]() ![]() ![]() ![]() ![]() | ||||||||
Relationships | |
Relationships |
Notes | |
(0026915) Brad King (manager) 2011-06-20 09:30 |
I've never really worked on CPack myself but from a quick glance I'll guess that the call to cmSystemTools::CopyFileIfDifferent in DoPackage is the culprit, and that it was originally added here: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3bce601c [^] (and was moved to its current location by your refactoring). |
(0026925) Eric NOULARD (developer) 2011-06-20 15:34 |
You mean something added in 2006 :-] I'm not sure to follow you at all. There is currently only two places where "cmSystemTools::CopyFileIfDifferent" appears now. In DoPackage this is used to copy package file itself so this cannot be the culprit. And this is used in cmCPackGenerator::InstallProjectViaInstalledDirectories where we check if "/* If it is not a symlink then do a plain copy */" before doing that. I'll have a look tonight, I'm in the middle of computer change so I may have mixed up my testing. |
(0026930) Eric NOULARD (developer) 2011-06-20 18:19 |
Ok tested againts current master (cmake version 2.8.4.20110620-ge85df) The problem is the same. I suspect some subtlities in the way symlinks are handled or detected by libarchive. Like I said before I am able to create TGZ with CPack which contains symlinks (at least between files in the same directory). in CMake/CPack libarchive is imported from upstream (http://code.google.com/p/libarchive [^]) into Utilities/cmlibarchive. Then CPack uses: Sources/cmArchiveWrite.* a file is added using method cmArchiveWrite::AddFile which does not seem to handle symlink at all!! (no reference to archive_entry_*_symlink) so I guess something is missing here and may be it appears not to be automatically handled for symlink NOT within the same path level. Just a wild guess I'm no lilbarchive expert. May be someone knowing libarchive better than me can have a look at this? |
(0026931) Brad King (manager) 2011-06-21 08:23 edited on: 2011-06-21 08:24 |
Using "cmake -E tar" works fine and that uses the same libarchive settings as cpack uses. The problem is that this is a *directory* symlink but CPack recursively globs and lists all *files* (perhaps in order to do component breakdown). In the posted example CPack asks libarchive to package these paths: cmt-1.1.1-Linux/bin/cmtdir/cmt cmt-1.1.1-Linux/cmtdir/cmt explicitly so it never actually inspects "cmtdir". |
(0026932) Brad King (manager) 2011-06-21 08:29 |
I think the solution is to fix the recursive file list collection to treat symlinks as files whether they point to directories or not. I'm not sure whether this will have any compatibility consequences. |
(0026933) Eric NOULARD (developer) 2011-06-21 08:53 |
Ok I see, CPack needed to know the list of file to package before component as well because each specific generator may do something specific with this list of files. [Group] Component packaging added an extra need. I do not currently have much spare time but passing symlinks in the filelist should be doable I'll try something like that when the time comes, I cannot ensure any deadline though. |
(0026943) Alan W. Irwin (reporter) 2011-06-21 16:41 |
I reported this as a bug so long ago that it predates your mantis system (!) and apparently that bug report was not propagated to that system (or at least I cannot find it now). I haven't done any recent tests of this, but in all cases symlinks should be preserved as symlinks in the generated tarball. My FreeEOS use case is a source tree with one or more very large files in it (in this case various tables of equation-of-state results) which are symlinked multiple times in my local source tree to save a ton of disk space. The symptom of the issue was a source tarball so large that I had to work around the problem by excluding the large files and asking my users to download those separately. Obviously, this is not a very satisfactory workaround so I would very much appreciate a fix for this problem. |
(0026945) Eric NOULARD (developer) 2011-06-22 02:33 |
Hi Alan, I totally agree no doubt it's a bug. The thing is I do not currently have time to work on it. Now if you have a test case which is somehow different then file a complement here that way it could be work out at the same time. I say that because you seem to be concerned by **SOURCE** tarball which is not handled strictly the same way by CPack (CPACK_INSTALLED_DIRECTORIES is used) so your testcase may be different from John's. |
(0027183) Johan Björk (reporter) 2011-08-08 13:46 |
Here is a very short example. I would love to up the severity of this bug, this makes it really hard to build proper OSX frameworks. #cmake ../ -- Configuring done -- Generating done -- Build files have been written to: /Users/phb/DEV/test/cmake-framework/build cherimoya:build phb$ cpack CPack: Create package using TGZ CPack: Install projects CPack: - Run preinstall target for: Project CPack: - Install project: Project CPack: Create package CPack: - package: /Users/phb/DEV/test/cmake-framework/build/test-0.0.1-Darwin.tar.gz generated. cherimoya:build phb$ tar -tvzf test-0.0.1-Darwin.tar.gz lrwxr-xr-x 0 phb staff 0 Aug 8 19:45 test-0.0.1-Darwin/foo.framework/foo -> Versions/Current/foo -rw-r--r-- 0 phb staff 0 Aug 8 19:45 test-0.0.1-Darwin/foo.framework/Headers/foo.h -rw-r--r-- 0 phb staff 732 Aug 8 19:44 test-0.0.1-Darwin/foo.framework/Resources/Info.plist -rwxr-xr-x 0 phb staff 4104 Aug 8 19:45 test-0.0.1-Darwin/foo.framework/Versions/5/foo -rw-r--r-- 0 phb staff 0 Aug 8 19:45 test-0.0.1-Darwin/foo.framework/Versions/5/Headers/foo.h -rw-r--r-- 0 phb staff 732 Aug 8 19:44 test-0.0.1-Darwin/foo.framework/Versions/5/Resources/Info.plist -rwxr-xr-x 0 phb staff 4104 Aug 8 19:45 test-0.0.1-Darwin/foo.framework/Versions/Current/foo -rw-r--r-- 0 phb staff 0 Aug 8 19:45 test-0.0.1-Darwin/foo.framework/Versions/Current/Headers/foo.h -rw-r--r-- 0 phb staff 732 Aug 8 19:44 test-0.0.1-Darwin/foo.framework/Versions/Current/Resources/Info.plist cherimoya:build phb$ tar -tvzf test-0.0.1-Darwin.tar.gz |
(0027184) Alan W. Irwin (reporter) 2011-08-08 23:20 |
To Eric: Sorry it took so long for me to reply to your request, but I now have a case that serves as a simple example that works for _source_ package creation. IOW, the old source package symlink bug that I mentioned appears to be fixed now. Here is simple test case for those who want to review why this works while binary packaging appears to have troubles with symlinks (from what others have said here). irwin@raven> ls -l total 12 -rw-r--r-- 1 irwin irwin 311 Aug 8 20:05 CMakeLists.txt lrwxrwxrwx 1 irwin irwin 14 Aug 8 20:02 CMakeLists.txt_symlink -> CMakeLists.txt -rw-r--r-- 1 irwin irwin 310 Aug 8 20:01 CMakeLists.txt~ drwxr-xr-x 4 irwin irwin 4096 Aug 8 20:03 build_dir/ irwin@raven> cat CMakeLists.txt cmake_minimum_required(VERSION 2.8.5 FATAL_ERROR) project(test_source_tarball NONE) set(CPACK_SOURCE_GENERATOR TGZ) set(CPACK_SOURCE_PACKAGE_FILE_NAME "test_source_tarball" CACHE INTERNAL "tarball basename" ) set(CPACK_SOURCE_IGNORE_FILES "~$" "^${PROJECT_SOURCE_DIR}/build_dir/" ) include(CPack) If you cd to build_dir, empty it with rm -rf, then run cmake .. make package_source then you generate a tarball with the expected symlink to CMakeLists.txt rather than a file copy like you got in the bad old days. Hope this helps. |
(0027377) David Cole (manager) 2011-09-06 14:47 |
The following series of commits was merged to 'next' today, in this order: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=98cb017a9deca35cd9a67e03b6bd95064b6d2fb7 [^] http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=527a40f06fc7f0ea6aa9c1fe96fb0fe5611fa633 [^] http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d78bdb27832c91c775ad3782c9eb436dcd6a1e7c [^] http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=7b8dcdd17312a7c3ed731743468136b0ea89a6c7 [^] These should be equivalent to the changes attaches as a patch series to this issue, but they had to be re-worked and re-ordered because the KWSys changes had to be committed via the KWSys cvs repository first. |
(0027378) David Cole (manager) 2011-09-06 14:48 |
See previous note for pointers to commits that fix this issue |
(0028824) David Cole (manager) 2012-03-06 08:37 |
Closing resolved issues that have not been updated in more than 4 months. |
Notes |
Issue History | |||
Date Modified | Username | Field | Change |
2011-06-17 16:57 | Eric NOULARD | New Issue | |
2011-06-17 16:57 | Eric NOULARD | File Added: cmt.tgz | |
2011-06-20 09:30 | Brad King | Note Added: 0026915 | |
2011-06-20 15:34 | Eric NOULARD | Note Added: 0026925 | |
2011-06-20 15:34 | Eric NOULARD | Assigned To | => Eric NOULARD |
2011-06-20 15:34 | Eric NOULARD | Status | new => assigned |
2011-06-20 18:19 | Eric NOULARD | Note Added: 0026930 | |
2011-06-20 18:19 | Eric NOULARD | Assigned To | Eric NOULARD => |
2011-06-21 08:23 | Brad King | Note Added: 0026931 | |
2011-06-21 08:24 | Brad King | Note Edited: 0026931 | |
2011-06-21 08:29 | Brad King | Note Added: 0026932 | |
2011-06-21 08:53 | Eric NOULARD | Note Added: 0026933 | |
2011-06-21 09:11 | Brad King | Status | assigned => backlog |
2011-06-21 16:41 | Alan W. Irwin | Note Added: 0026943 | |
2011-06-22 02:33 | Eric NOULARD | Note Added: 0026945 | |
2011-08-08 13:46 | Johan Björk | Note Added: 0027183 | |
2011-08-08 13:47 | Johan Björk | File Added: CMakeLists.txt | |
2011-08-08 23:20 | Alan W. Irwin | Note Added: 0027184 | |
2011-08-27 13:11 | Johan Björk | File Added: 0001-Fix-12284-Symlinks-are-not-preserved-in-CPack-tar.gz.patch | |
2011-08-27 13:12 | Johan Björk | Note Added: 0027275 | |
2011-08-27 13:13 | Johan Björk | Note Deleted: 0027275 | |
2011-08-27 13:41 | Johan Björk | File Added: 0002-kwsys-Glob-Fix-so-directory-symlinks-are-added-as-fi.patch | |
2011-08-27 13:41 | Johan Björk | File Added: 0003-cpack-Fix-12284-don-t-recurse-through-directory-syml.patch | |
2011-08-27 13:41 | Johan Björk | File Added: 0001-ArchiveWrite-Don-t-recurse-through-directory-symlink.patch | |
2011-09-06 14:15 | David Cole | Assigned To | => David Cole |
2011-09-06 14:15 | David Cole | Status | backlog => assigned |
2011-09-06 14:16 | David Cole | Target Version | => CMake 2.8.6 |
2011-09-06 14:47 | David Cole | Note Added: 0027377 | |
2011-09-06 14:48 | David Cole | Note Added: 0027378 | |
2011-09-06 14:48 | David Cole | Status | assigned => resolved |
2011-09-06 14:48 | David Cole | Fixed in Version | => CMake 2.8.6 |
2011-09-06 14:48 | David Cole | Resolution | open => fixed |
2012-03-06 08:37 | David Cole | Note Added: 0028824 | |
2012-03-06 08:37 | David Cole | Status | resolved => closed |
Issue History |
Copyright © 2000 - 2018 MantisBT Team |