[cmake-developers] [CMake 0013251]: cmake -E tar <file> fails to extract some files correctly
Mantis Bug Tracker
mantis at public.kitware.com
Fri Dec 7 11:12:58 EST 2012
The following issue is now in status NEW (again)
======================================================================
http://www.cmake.org/Bug/view.php?id=13251
======================================================================
Reported By: sigma
Assigned To:
======================================================================
Project: CMake
Issue ID: 13251
Category: CMake
Reproducibility: always
Severity: major
Priority: normal
Status: new
======================================================================
Date Submitted: 2012-05-27 07:37 EDT
Last Modified: 2012-12-07 11:12 EST
======================================================================
Summary: cmake -E tar <file> fails to extract some files
correctly
Description:
On certain tar.gz files cmake fails to extract them properly. I suspect this has
to do with tar.gz files that have unix symlinks and are extracted on windows.
One such file can be obtained from:
http://openssl.org/source/openssl-1.0.1c.tar.gz
When I try: cmake -E tar xf openssl-1.0.1c.tar.gz it fails on some files:
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/apps/md4.c
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/aes.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/asn1.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/asn1_mac.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/asn1t.h
CMake Error: Problem with archive_write_header(): Can't create '\'
CMake Error: Current file:openssl-1.0.1c/include/openssl/bio.h
CMake Error: Problem with archive_write_header(): Can't create '\'
Using other extraction tools, e.g. cygwin or 7zip, this works without any
problems.
======================================================================
Relationships ID Summary
----------------------------------------------------------------------
related to 0011176 "cmake -E tar" encodes long paths such ...
related to 0010162 CPack Cannot create binary zip archive ...
parent of 0011766 CPack using native TAR with adjustable ...
======================================================================
----------------------------------------------------------------------
(0029571) Eric NOULARD (developer) - 2012-05-27 07:56
http://www.cmake.org/Bug/view.php?id=13251#c29571
----------------------------------------------------------------------
I can extract this archive with CMake 2.8.8
on Linux (Debian Squeeze) without trouble...
----------------------------------------------------------------------
(0029572) Eric NOULARD (developer) - 2012-05-27 07:58
http://www.cmake.org/Bug/view.php?id=13251#c29572
----------------------------------------------------------------------
You are right those files are symlinks.
How do other tools like 7-zip handles those symlink on Windows?
Do they create plain files which are a copy of the symlink target?
Do they handle symlink in some way?
----------------------------------------------------------------------
(0029573) sigma (reporter) - 2012-05-27 10:48
http://www.cmake.org/Bug/view.php?id=13251#c29573
----------------------------------------------------------------------
I created a test tar.gz with various file types to see what happens.
Compress it on a linux box with tar (GNU tar) 1.26:
tar -cpvzf cmake-tar-test.tar.gz cmake-tar-test/
The archive contains:
cmake-tar-test/:
total 12
drwxr-xr-x 2 root root 4096 May 27 16:44 bin
drwxr-xr-x 2 root root 4096 May 27 17:23 links
drwxr-xr-x 2 root root 4096 May 27 16:44 src
cmake-tar-test/bin:
total 100
-rwxr--r-- 1 root root 99328 May 27 16:43 main.exe
cmake-tar-test/links:
total 0
lrwxrwxrwx 1 root root 6 May 27 16:46 dir-link -> ../src
lrwxrwxrwx 1 root root 15 May 27 16:45 main.cpp -> ../src/main.cpp
lrwxrwxrwx 1 root root 15 May 27 16:45 main.exe -> ../bin/main.exe
lrwxrwxrwx 1 root root 13 May 27 16:45 main.h -> ../src/main.h
cmake-tar-test/src:
total 8
-rwxr--r-- 1 root root 127 May 27 16:42 main.cpp
-rwxr--r-- 1 root root 19 May 27 16:42 main.h
So there is a mixture of symlinks to various filetypes and directories. main.exe
was compiled on win7 and is a valid executable.
I moved the files back to the Win7 machine to see how the different programs
(7-zip, cygwin, cmake) behave. cmake log is here:
----------------------------------------------------------------------------------------
cmake -E tar xvfz cmake-tar-test.tar.gz
x cmake-tar-test/
x cmake-tar-test/links/
x cmake-tar-test/links/main.cppCMake Error: Problem with archive_write_header():
Can't create '\'
CMake Error: Current file:cmake-tar-test/links/main.cpp
x cmake-tar-test/links/main.exeCMake Error: Problem with archive_write_header():
Can't create '\'
CMake Error: Current file:cmake-tar-test/links/main.exe
x cmake-tar-test/links/main.hCMake Error: Problem with archive_write_header():
Can't create '\'
CMake Error: Current file:cmake-tar-test/links/main.h
x cmake-tar-test/links/dir-linkCMake Error: Problem with archive_write_header():
Can't create '\'
CMake Error: Current file:cmake-tar-test/links/dir-link
x cmake-tar-test/src/
x cmake-tar-test/src/main.cpp
x cmake-tar-test/src/main.h
x cmake-tar-test/bin/
x cmake-tar-test/bin/main.exe
----------------------------------------------------------------------------------------
Here is what each program does:
original | 7zip
----------------------------------------------------------------------------------------
links/main.cpp | just contains the text ../src/main.cpp (NOT the contents of)
links/main.exe | just contains the text ../bin/main.exe (NOT the contents of)
links/main.h | just contains the text ../src/main.h (NOT the contents of)
links/dir-link | just contains the text ../src
----------------------------------------------------------------------------------------
original | cygwin
----------------------------------------------------------------------------------------
links/main.cpp | binary - contains "!<symlink>ÿþ....." (most likely a posix
link?)
links/main.exe | binary - contains "!<symlink>ÿþ....."
links/main.h | binary - contains "!<symlink>ÿþ....."
links/dir-link | binary - contains "!<symlink>ÿþ....."
----------------------------------------------------------------------------------------
original | cmake
----------------------------------------------------------------------------------------
links/main.cpp | link missing
links/main.exe | link missing
links/main.h | link missing
links/dir-link | link missing
----------------------------------------------------------------------------------------
I think there is no simple answer, since the output of all is typically not
useful. I didn't look closely, but the openssl package does compile and test
fine both if extracted with cygwin and 7zip, but it doesn't work with cmake,
because it doesn't extract the links at all.
----------------------------------------------------------------------
(0029577) Brad King (manager) - 2012-05-28 09:49
http://www.cmake.org/Bug/view.php?id=13251#c29577
----------------------------------------------------------------------
Possibly related upstream libarchive issue:
http://code.google.com/p/libarchive/issues/detail?id=259
----------------------------------------------------------------------
(0029578) Brad King (manager) - 2012-05-28 09:51
http://www.cmake.org/Bug/view.php?id=13251#c29578
----------------------------------------------------------------------
A quick experiment tells me that GNU tar on Windows (MinGW/MSYS) extracts a copy
of the content from the linked file, at least if the symlink points to another
entry in the tarball.
In my understanding the "symlink" generated by GNU tar under Cygwin is just a
file that Cygwin's runtime library recognizes and treats as a symlink for the
purposes of its POSIX emulation.
----------------------------------------------------------------------
(0029592) sigma (reporter) - 2012-05-31 18:47
http://www.cmake.org/Bug/view.php?id=13251#c29592
----------------------------------------------------------------------
Actually, just copying the contents of the pointed file would most likely enable
source packages to function properly most of the times afterwards, at the
expense of space. It is better than failing, like it does now.
Right now, I use ExternalProject to download and build openssl on windows as a
dependency for another project, this fails of course, as EP uses the internal
tar by default. I haven't done it yet, but I remember there is a way to specify
the command to use to extract it, so one can skip the internal tar. I need to
look into it, but it would be greatly appreciated if you could make it copy the
files.
----------------------------------------------------------------------
(0029593) sigma (reporter) - 2012-05-31 20:55
http://www.cmake.org/Bug/view.php?id=13251#c29593
----------------------------------------------------------------------
I spend some time debugging cmake and it seems this is purely because of the
failure of libarchive to handle symlinks.
----------------------------------------------------------------------
(0029594) Eric NOULARD (developer) - 2012-06-01 02:57
http://www.cmake.org/Bug/view.php?id=13251#c29594
----------------------------------------------------------------------
Hi,
Have a look at bug http://www.cmake.org/Bug/view.php?id=10162 you'll find that
libarchive *is* handling symlinks
but only on supported platforms and/or archive format.
See the libarchive isssue reference pointed by Brad as well.
In fact we do some tuning/verification when creating archive with libarchive
see cmArchiveWrite.[h|cxx]
but we do minimal error checking when extracting
see cmSystemTools.cxx
A) We could (theoretically) check, when extracting, whether if the entry is a
symlink and we are on Windows platform, in order to create a plain file
out of the symlink found in libarchive.
This could be seen as workaround of the inability of libarchive to do it
but it could be done.
B) Another option would be to error out when symlink is found in archive when
extracting on Windows.
C) Or put some work in order to make this fixed upstream.
I personnally think B) is not satisfactory, so that we should do A) and
possibly suppress it when C) is ready.
My 2 cents, I won't have time to work on this anyway.
----------------------------------------------------------------------
(0030431) David Cole (manager) - 2012-08-11 21:42
http://www.cmake.org/Bug/view.php?id=13251#c30431
----------------------------------------------------------------------
Sending old, never assigned issues to the backlog.
(The age of the bug, plus the fact that it's never been assigned to anyone means
that nobody is actively working on it...)
If an issue you care about is sent to the backlog when you feel it should have
been addressed in a different manner, please bring it up on the CMake mailing
list for discussion. Sign up for the mailing list here, if you're not already on
it: http://www.cmake.org/mailman/listinfo/cmake
It's easy to re-activate a bug here if you can find a CMake developer who has
the bandwidth to take it on, and ferry a fix through to our 'next' branch for
dashboard testing.
----------------------------------------------------------------------
(0031848) Kevin Burge (reporter) - 2012-12-07 09:15
http://www.cmake.org/Bug/view.php?id=13251#c31848
----------------------------------------------------------------------
I am seeing this same problem with the same package (openssl) with 2.8.10.2. I
agree that making the extract copy of the file is better than failing. My
options are: extract and repackage the gz with these links either removed
altogether or manually replaced with the original file. It seems that 2.8.10.2
is even worse than 2.8.8. At least it allowed you to continue building. Now it
fails and stops.
Please address this.
Issue History
Date Modified Username Field Change
======================================================================
2012-05-27 07:37 sigma New Issue
2012-05-27 07:52 Eric NOULARD Relationship added related to 0011766
2012-05-27 07:52 Eric NOULARD Relationship added related to 0011176
2012-05-27 07:52 Eric NOULARD Relationship deleted related to 0011766
2012-05-27 07:53 Eric NOULARD Relationship added parent of 0011766
2012-05-27 07:56 Eric NOULARD Note Added: 0029571
2012-05-27 07:58 Eric NOULARD Note Added: 0029572
2012-05-27 10:48 sigma File Added: cmake-tar-test.tar.gz
2012-05-27 10:48 sigma Note Added: 0029573
2012-05-28 09:49 Brad King Note Added: 0029577
2012-05-28 09:51 Brad King Note Added: 0029578
2012-05-31 18:47 sigma Note Added: 0029592
2012-05-31 20:55 sigma Note Added: 0029593
2012-06-01 02:39 Eric NOULARD Relationship added related to 0010162
2012-06-01 02:56 Eric NOULARD Note Added: 0029594
2012-06-01 02:57 Eric NOULARD Note Edited: 0029594
2012-08-11 21:42 David Cole Status new => backlog
2012-08-11 21:42 David Cole Note Added: 0030431
2012-12-07 09:11 Kevin Burge Issue Monitored: Kevin Burge
2012-12-07 09:15 Kevin Burge Note Added: 0031848
2012-12-07 11:11 David Cole Assigned To => David Cole
2012-12-07 11:11 David Cole Status backlog => assigned
2012-12-07 11:12 David Cole Assigned To David Cole =>
2012-12-07 11:12 David Cole Status assigned => new
======================================================================
More information about the cmake-developers
mailing list