[cmake-developers] Unexpected bad status for file DOWNLOAD for http://*.xz URLs [SOLVED]

[Alan W. Irwin]

On 2013-07-11 16:30-0700 Alan W. Irwin wrote:

> These commands
>
> file(DOWNLOAD
> "http://download.gnome.org/sources/glib/2.32/glib-2.32.1.tar.xz"
> "/home/software/plplot_svn/HEAD/plplot_allura/cmake/build_projects/cmake_downloaded_glib-2.32.1.tar.xz"
> SHOW_PROGRESS STATUS status)
> message(STATUS "Download status = ${status}")
>
> instantly give me the following results:
>
> -- [download 0% complete]
> -- Download status = 1;"unsupported protocol"
>
> with a zero-length cmake_downloaded_glib-2.32.1.tar.xz produced.

I used gdb to attempt to debug this issue (with essentially zero
knowledge of the curl code so this was fairly difficult for me). 
Eventually I traced
the issue to line

3026      else if (strequal(conn->protostr, "HTTPS")) {

in (CMake-2.8.10.2) Utilities/cmcurl/url.c

(gdb) print conn->protostr
$20 = "https\000\000\000\000\000\000\000\000\000\000"

I believe protostr is https (rather than http as in the above URL)
because the redirects (as shown by wget and also curl without the -L
option) are from

http://download.gnome.org ==> https://download.gnome.org ==> http://ftp.gnome.org/pub/GNOME

and it appears the cmake curl code is choking on that https protocol
(at least for my CMake version which I built with the default
bootstrap method on Linux).  The next lines in url.c after 3026 are

#if defined(USE_SSL) && !defined(CURL_DISABLE_HTTP)

and that macro is false for the default bootstrap build
so it skips to the following lines

     failf(data, LIBCURL_NAME
           " was built with SSL disabled, https: not supported!");
     return CURLE_UNSUPPORTED_PROTOCOL;

That must mean that SSL is disabled for cmake downloads
(even if via a http redirect as in this case) 
for my bootstrap build of cmake-2.8.10.2

So I tried a new bootstrap build (this time using the option
--system-curl).  That found _a_ system curl library,
libcurl-gnutls.so, and the build went fine with no errors.  However,
now the symptom from the above test case is

software at raven> cmake -P test.cmake
-- [download 0% complete]
-- Download status = 35;"SSL connect error"

So after some thrashing around I installed the Debian wheezy
libcurl4-openssl-dev package (which removed the libcurl4-gnutls-dev
package that had been there before). Again the bootstrap build went
well (this time finding the libcurl.so library rather than
libcurl-gnutls.so), and the result of the above command finally
suceeded with [...]
-- [download 100% complete]
-- Download status = 0;"No error"

So all is well that ends well, but still there are two issues that
really bother me from this experience.

1. The CMake in-house curl library that you get by default should
allow use of the https protocol.  Otherwise, it is not that useful
since https has widespread use these days (even through redirects as
for the above http URL). Could in-house openssl be implemented by
simply making an in-house openssl library part of the CMake build
(much like how the in-house libcurl library is currently handled)?

2. Why didn't the system libcurl-gnutls.so library give a good result?
I am fairly sure that is a fundamentally sound library (since the
gnutls version tends to be more hyped than the openssl version).  Is
the problem that CMake is just not setting up use of libcurl-gnutls.so
properly?

Because of these two issues, it has been quite a struggle to configure
the CMake build and install the correct system packages to create a
build that gives reliable downloads with CMake.  So I hope
the above two issues get addressed.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________