[cmake-developers] Unexpected bad status for file DOWNLOAD for http://*.xz URLs

Thu Jul 11 19:48:32 EDT 2013

Hello,

Related to this, I implemented an EXTRACT_COMMAND option to
ExternalProject_Add which would enable to download anything, provided that
an EXTRACT_COMMAND is also passed so that CMake knows how to extract what
it downloaded. It is specially useful for Windows, where NSIS,
InstallShield, etc can be extracted silently but they all have the .exe /
.msi extension.

It's available from here:

https://elpauer.assembla.com/code/elpauercmake/git/nodes/epa_extract_command

Branch epa_extract_command

(that clone is a bit outdated, btw)

It's used like this:

      ExternalProject_Add( graphicsmagick
	    DOWNLOAD_NAME graphicsmagick.exe
	    DOWNLOAD_DIR ${WINST_BASEDIR}/downloads
	    URL ${graphicsmagick_URL}
	    URL_MD5 ${graphicsmagick_MD5}
	    *EXTRACT_COMMAND ${CMAKE_COMMAND} -E copy
${WINST_BATDIR}/patches/graphicsmagick.exe.manifest
${WINST_PREFIX}/bin/*
	    LOG_DOWNLOAD 1
	    LOG_UPDATE 1
	    LOG_CONFIGURE 1
	    LOG_BUILD 1
	    LOG_TEST 1
	    LOG_INSTALL 1
	    )

On Fri, Jul 12, 2013 at 1:30 AM, Alan W. Irwin <irwin at beluga.phys.uvic.ca>wrote:

> These commands
>
> file(DOWNLOAD
> "http://download.gnome.org/**sources/glib/2.32/glib-2.32.1.**tar.xz<http://download.gnome.org/sources/glib/2.32/glib-2.32.1.tar.xz>
> "
> "/home/software/plplot_svn/**HEAD/plplot_allura/cmake/**
> build_projects/cmake_**downloaded_glib-2.32.1.tar.xz"
> SHOW_PROGRESS STATUS status)
> message(STATUS "Download status = ${status}")
>
> instantly give me the following results:
>
> -- [download 0% complete]
> -- Download status = 1;"unsupported protocol"
>
> with a zero-length cmake_downloaded_glib-2.32.1.**tar.xz produced.
>
> Yet when I use wget to obtain
> http://download.gnome.org/**sources/glib/2.32/glib-2.32.1.**tar.xz<http://download.gnome.org/sources/glib/2.32/glib-2.32.1.tar.xz>there
> are no problems.  i.e.,
>
> software at raven> wget http://download.gnome.org/**
> sources/glib/2.32/glib-2.32.1.**tar.xz<http://download.gnome.org/sources/glib/2.32/glib-2.32.1.tar.xz>
> --2013-07-11 16:04:21--  http://download.gnome.org/**
> sources/glib/2.32/glib-2.32.1.**tar.xz<http://download.gnome.org/sources/glib/2.32/glib-2.32.1.tar.xz>
> Resolving download.gnome.org (download.gnome.org)... 209.132.180.168,
> 209.132.180.180
> Connecting to download.gnome.org (download.gnome.org)|209.132.**180.168|:80...
> connected.
> HTTP request sent, awaiting response... 302 Found
> Location: https://download.gnome.org/**sources/glib/2.32/glib-2.32.1.**
> tar.xz <https://download.gnome.org/sources/glib/2.32/glib-2.32.1.tar.xz>[following]
> --2013-07-11 16:04:21--  https://download.gnome.org/**
> sources/glib/2.32/glib-2.32.1.**tar.xz<https://download.gnome.org/sources/glib/2.32/glib-2.32.1.tar.xz>
> Connecting to download.gnome.org (download.gnome.org)|209.132.**180.168|:443...
> connected.
> HTTP request sent, awaiting response... 301 Moved Permanently
> Location: http://ftp.gnome.org/pub/**GNOME/sources/glib/2.32/glib-**
> 2.32.1.tar.xz<http://ftp.gnome.org/pub/GNOME/sources/glib/2.32/glib-2.32.1.tar.xz>[following]
> --2013-07-11 16:04:22--  http://ftp.gnome.org/pub/**
> GNOME/sources/glib/2.32/glib-**2.32.1.tar.xz<http://ftp.gnome.org/pub/GNOME/sources/glib/2.32/glib-2.32.1.tar.xz>
> Resolving ftp.gnome.org (ftp.gnome.org)... 130.239.18.137,
> 130.239.18.138, 130.239.18.163, ...
> Connecting to ftp.gnome.org (ftp.gnome.org)|130.239.18.**137|:80...
> connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 6138200 (5.9M) [application/x-xz]
> Saving to: `glib-2.32.1.tar.xz.1'
>
> 100%[=========================**==============================**====================>]
> 6,138,200    956K/s   in 12s
>
> 2013-07-11 16:04:35 (508 KB/s) - `glib-2.32.1.tar.xz.1' saved
> [6138200/6138200]
>
>
> If I do the equivalent CMake commands above for some random *.tar.gz
> the result was
>
> -- [download 25% complete]
> -- [download 100% complete]
> -- Download status = 0;"no error"
>
> with a full file downloaded.
>
> So I don't think I have any syntax error in the above CMake logic.
>
> One explanation is CMake might be having trouble with the http
> protocol when there is a tar.xz suffix on the URL.  To check that
> hypothesis, I also tried downloading
> http://download.gnome.org/**sources/gcr/3.4/gcr-3.4.1.tar.**xz<http://download.gnome.org/sources/gcr/3.4/gcr-3.4.1.tar.xz>and got the
> same bad status.
>
> But why should CMake care in the slightest about the tar.xz suffix on the
> file that is downloaded?  Shouldn't it just download the file come
> hell or high water?
>
> BTW, I also just discovered that cmake has no trouble unpacking a
> *.tar.xz file using, e.g.,
>
> cmake -E tar xfz glib-2.32.1.tar.xz
>
> I also found that some small changes had to be made to
> ExternalProjects.cmake to allow files of the form *.tar.xz to be
> processed further (e.g., by the above cmake -E tar command).  I plan
> to submit that patch to your bug tracker.  But aside from that one
> issue (fixed with a patch) and the above strange issue with *.tar.xz
> files that cannot be downloaded with file(DOWNLOAD), the rest of my
> use of ExternalProject.cmake should be okay for build_projects (where
> I have discovered that many downloadable source tarballs are in the
> *.tar.xz form).
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state
> implementation for stellar interiors (freeeos.sf.net); the Time
> Ephemerides project (timeephem.sf.net); PLplot scientific plotting
> software package (plplot.sf.net); the libLASi project
> (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
> and the Linux Brochure Project (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/**
> opensource/opensource.html<http://www.kitware.com/opensource/opensource.html>
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/**CMake_FAQ<http://www.cmake.org/Wiki/CMake_FAQ>
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/cgi-**bin/mailman/listinfo/cmake-**developers<http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers>
>

-- 
Pau Garcia i Quiles
http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20130712/376d5cb1/attachment.html>