[CMake] file(DOWNLOAD) and HTTP 302 (redirect)

Fri Jun 4 11:08:03 EDT 2010

I don't know what options are compiled in for CMake, but it looks like
this is the libcurl option required to follow redirects:

CURLOPT_FOLLOWLOCATION 

A parameter set to 1 tells the library to follow any Location: header
that the server sends as part of an HTTP header. 

This means that the library will re-send the same request on the new
location and follow new Location: headers all the way until no more such
headers are returned. CURLOPT_MAXREDIRS can be used to limit the number
of redirects libcurl will follow. 

Since 7.19.4, libcurl can limit what protocols it will automatically
follow. The accepted protocols are set with CURLOPT_REDIR_PROTOCOLS and
it excludes the FILE protocol by default.

Aaron C. Meadows 

________________________________

From: cmake-bounces at cmake.org [mailto:cmake-bounces at cmake.org] On Behalf
Of David Cole
Sent: Friday, June 04, 2010 9:47 AM
To: Michael Wild
Cc: CMake List
Subject: Re: [CMake] file(DOWNLOAD) and HTTP 302 (redirect)

On Fri, Jun 4, 2010 at 8:03 AM, Michael Wild <themiwi at gmail.com> wrote:

The log looks like this (I've put a message("log = ${log}") into
ExternalProject.cmake to get this, since the numeric error code is 0):

<snip>
-- downloading...

src='http://downloads.sourceforge.net/project/freefoam/ThirdParty/zlib/z
lib-1.2.5.tar.gz'

dst='/Users/mwild/Projects/FreeFOAM/build-pu/ThirdParty/zlib/src/zlib-1.
2.5.tar.gz'
    timeout='none'
log = About to connect() to downloads.sourceforge.net port 80 (#0)
 Trying 216.34.181.59... connected
Connected to downloads.sourceforge.net (216.34.181.59) port 80 (#0)
GET /project/freefoam/ThirdParty/zlib/zlib-1.2.5.tar.gz HTTP/1.1
Host: downloads.sourceforge.net
Accept: */*

HTTP/1.1 302 Found
X-Powered-By: PHP/5.2.9
Content-Disposition: attachment; filename="zlib-1.2.5.tar.gz"
Location:
http://surfnet.dl.sourceforge.net/project/freefoam/ThirdParty/zlib/zlib-
1.2.5.tar.gz
Content-type
<http://surfnet.dl.sourceforge.net/project/freefoam/ThirdParty/zlib/zlib
-1.2.5.tar.gz%0d%0aContent-type> : text/html
Content-Length: 0
Date: Fri, 04 Jun 2010 07:52:13 GMT
Server: lighttpd/1.4.26

Connection #0 to host downloads.sourceforge.net left intact
Closing connection #0

-- downloading... done
</snip>

So you see, CMake really thinks this succeeded.

Hmm. I wonder if there's some sort of curl option we should be passing
to get redirects to work automatically...?

	I really am looking forward to the EXPECTED_MD5 feature, please
also make it available through ExternalProject_Add! Currently I'm doing
this manually via an additional step, but it's awfully fragile...

It is. That was actually the main motivation for the additional
file(DOWNLOAD arg... In CMake 'next' there's a URL_MD5 arg to
ExternalProject_Add now.

	Michael

	On 4. Jun, 2010, at 13:57 , David Cole wrote:

	> I'm pretty sure the original intent was for file(DOWNLOAD ...)
to not raise
	> errors so that scripts can analyze status and log and decide
what to do next
	> rather than forcing a CMake error...
	>
	> I recently added EXPECTED_MD5 to file(DOWNLOAD in the 'next'
branch of
	> CMake. If a download fails when there's an expected MD5 sum
given, then it
	> *does* raise an error. Perhaps you'll find that useful in the
next
	> release...
	>
	> Cheers,
	> David C.
	>
	>
	> On Fri, Jun 4, 2010 at 7:55 AM, David Cole
<david.cole at kitware.com> wrote:
	>
	>> What if you do something this...? What do status and log tell
you?
	>>
	>> set(remote "http://......")
	>> set(local "some file name")
	>>
	>> file(DOWNLOAD
	>>
	>>  "${remote}"
	>>
	>>  "${local}"
	>>
	>>  STATUS status
	>>
	>>  LOG log)
	>>
	>>
	>> list(GET status 0 status_code)
	>>
	>> list(GET status 1 status_string)
	>>
	>>
	>> if(NOT status_code EQUAL 0)
	>>
	>>  message(FATAL_ERROR "error: downloading '${remote}' failed
	>>
	>>  status_code: ${status_code}
	>>
	>>  status_string: ${status_string}
	>>
	>>  log: ${log}
	>>
	>> ")
	>>
	>> endif()
	>>
	>>
	>>
	>> On Fri, Jun 4, 2010 at 4:00 AM, Michael Wild
<themiwi at gmail.com> wrote:
	>>
	>>> When downloading something from
http://downloads.sourceforge.net/. <http://downloads.sourceforge.net/>
.. you
	>>> always get a 302 redirect to one of the many mirror servers.
Unfortunately,
	>>> it seems that file(DOWNLOAD ...) doesn't follow the
redirect, but just stops
	>>> and leaves an empty file behind. No error, no nothing. Is
this intentional?
	>>>
	>>> I'd rather not use a specific mirror...
	>>>
	>>> Michael
	>>> _______________________________________________
	>>> Powered by www.kitware.com
	>>>
	>>> Visit other Kitware open-source projects at
	>>> http://www.kitware.com/opensource/opensource.html
	>>>
	>>> Please keep messages on-topic and check the CMake FAQ at:
	>>> http://www.cmake.org/Wiki/CMake_FAQ
	>>>
	>>> Follow this link to subscribe/unsubscribe:
	>>> http://www.cmake.org/mailman/listinfo/cmake
	>>>
	>>
	>>

This email was sent to you by Thomson Reuters, the global news and information company.
Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20100604/fe100bb3/attachment-0001.htm>