[CMake] file(DOWNLOAD) and HTTP 302 (redirect)

Michael Wild themiwi at gmail.com
Fri Jun 4 11:38:41 EDT 2010


The following patch does work for me:

diff --git a/Source/cmFileCommand.cxx b/Source/cmFileCommand.cxx
index 5611527..3b18e0b 100644
--- a/Source/cmFileCommand.cxx
+++ b/Source/cmFileCommand.cxx
@@ -2629,6 +2629,14 @@ cmFileCommand::HandleDownloadCommand(std::vector<std::string>
     return false;
     }
 
+  res = ::curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
+  if (res != CURLE_OK)
+    {
+    std::string errstring = "FILE(DOWNLOAD ) error; cannot set follow-redirect option: ";
+    errstring += ::curl_easy_strerror(res);
+    return false;
+    }
+
   if(verboseLog.size())
     {
     res = ::curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
--

However, I didn't test it thoroughly, and am not sure whether it would be a good idea to restrict the protocols and number of redirects.

Michael

On 4. Jun, 2010, at 17:08 , <aaron.meadows at thomsonreuters.com> <aaron.meadows at thomsonreuters.com> wrote:

> I don't know what options are compiled in for CMake, but it looks like
> this is the libcurl option required to follow redirects:
> 
> 
> 
> CURLOPT_FOLLOWLOCATION 
> 
> 
> 
> A parameter set to 1 tells the library to follow any Location: header
> that the server sends as part of an HTTP header. 
> 
> 
> 
> This means that the library will re-send the same request on the new
> location and follow new Location: headers all the way until no more such
> headers are returned. CURLOPT_MAXREDIRS can be used to limit the number
> of redirects libcurl will follow. 
> 
> 
> 
> Since 7.19.4, libcurl can limit what protocols it will automatically
> follow. The accepted protocols are set with CURLOPT_REDIR_PROTOCOLS and
> it excludes the FILE protocol by default.
> 
> 
> 
> Aaron C. Meadows 
> 
> ________________________________
> 
> From: cmake-bounces at cmake.org [mailto:cmake-bounces at cmake.org] On Behalf
> Of David Cole
> Sent: Friday, June 04, 2010 9:47 AM
> To: Michael Wild
> Cc: CMake List
> Subject: Re: [CMake] file(DOWNLOAD) and HTTP 302 (redirect)
> 
> 
> 
> On Fri, Jun 4, 2010 at 8:03 AM, Michael Wild <themiwi at gmail.com> wrote:
> 
> The log looks like this (I've put a message("log = ${log}") into
> ExternalProject.cmake to get this, since the numeric error code is 0):
> 
> <snip>
> -- downloading...
> 
> src='http://downloads.sourceforge.net/project/freefoam/ThirdParty/zlib/z
> lib-1.2.5.tar.gz'
> 
> dst='/Users/mwild/Projects/FreeFOAM/build-pu/ThirdParty/zlib/src/zlib-1.
> 2.5.tar.gz'
>    timeout='none'
> log = About to connect() to downloads.sourceforge.net port 80 (#0)
> Trying 216.34.181.59... connected
> Connected to downloads.sourceforge.net (216.34.181.59) port 80 (#0)
> GET /project/freefoam/ThirdParty/zlib/zlib-1.2.5.tar.gz HTTP/1.1
> Host: downloads.sourceforge.net
> Accept: */*
> 
> HTTP/1.1 302 Found
> X-Powered-By: PHP/5.2.9
> Content-Disposition: attachment; filename="zlib-1.2.5.tar.gz"
> Location:
> http://surfnet.dl.sourceforge.net/project/freefoam/ThirdParty/zlib/zlib-
> 1.2.5.tar.gz
> Content-type
> <http://surfnet.dl.sourceforge.net/project/freefoam/ThirdParty/zlib/zlib
> -1.2.5.tar.gz%0d%0aContent-type> : text/html
> Content-Length: 0
> Date: Fri, 04 Jun 2010 07:52:13 GMT
> Server: lighttpd/1.4.26
> 
> Connection #0 to host downloads.sourceforge.net left intact
> Closing connection #0
> 
> -- downloading... done
> </snip>
> 
> So you see, CMake really thinks this succeeded.
> 
> 
> 
> Hmm. I wonder if there's some sort of curl option we should be passing
> to get redirects to work automatically...?
> 
> 
> 
> 	
> 	I really am looking forward to the EXPECTED_MD5 feature, please
> also make it available through ExternalProject_Add! Currently I'm doing
> this manually via an additional step, but it's awfully fragile...
> 
> 
> 
> It is. That was actually the main motivation for the additional
> file(DOWNLOAD arg... In CMake 'next' there's a URL_MD5 arg to
> ExternalProject_Add now.
> 
> 
> 
> 
> 
> 	
> 	Michael
> 
> 	
> 	On 4. Jun, 2010, at 13:57 , David Cole wrote:
> 	
> 	> I'm pretty sure the original intent was for file(DOWNLOAD ...)
> to not raise
> 	> errors so that scripts can analyze status and log and decide
> what to do next
> 	> rather than forcing a CMake error...
> 	>
> 	> I recently added EXPECTED_MD5 to file(DOWNLOAD in the 'next'
> branch of
> 	> CMake. If a download fails when there's an expected MD5 sum
> given, then it
> 	> *does* raise an error. Perhaps you'll find that useful in the
> next
> 	> release...
> 	>
> 	> Cheers,
> 	> David C.
> 	>
> 	>
> 	> On Fri, Jun 4, 2010 at 7:55 AM, David Cole
> <david.cole at kitware.com> wrote:
> 	>
> 	>> What if you do something this...? What do status and log tell
> you?
> 	>>
> 	>> set(remote "http://......")
> 	>> set(local "some file name")
> 	>>
> 	>> file(DOWNLOAD
> 	>>
> 	>>  "${remote}"
> 	>>
> 	>>  "${local}"
> 	>>
> 	>>  STATUS status
> 	>>
> 	>>  LOG log)
> 	>>
> 	>>
> 	>> list(GET status 0 status_code)
> 	>>
> 	>> list(GET status 1 status_string)
> 	>>
> 	>>
> 	>> if(NOT status_code EQUAL 0)
> 	>>
> 	>>  message(FATAL_ERROR "error: downloading '${remote}' failed
> 	>>
> 	>>  status_code: ${status_code}
> 	>>
> 	>>  status_string: ${status_string}
> 	>>
> 	>>  log: ${log}
> 	>>
> 	>> ")
> 	>>
> 	>> endif()
> 	>>
> 	>>
> 	>>
> 	>> On Fri, Jun 4, 2010 at 4:00 AM, Michael Wild
> <themiwi at gmail.com> wrote:
> 	>>
> 	>>> When downloading something from
> http://downloads.sourceforge.net/. <http://downloads.sourceforge.net/>
> .. you
> 	>>> always get a 302 redirect to one of the many mirror servers.
> Unfortunately,
> 	>>> it seems that file(DOWNLOAD ...) doesn't follow the
> redirect, but just stops
> 	>>> and leaves an empty file behind. No error, no nothing. Is
> this intentional?
> 	>>>
> 	>>> I'd rather not use a specific mirror...
> 	>>>
> 	>>> Michael
> 	>>> _______________________________________________
> 	>>> Powered by www.kitware.com
> 	>>>
> 	>>> Visit other Kitware open-source projects at
> 	>>> http://www.kitware.com/opensource/opensource.html
> 	>>>
> 	>>> Please keep messages on-topic and check the CMake FAQ at:
> 	>>> http://www.cmake.org/Wiki/CMake_FAQ
> 	>>>
> 	>>> Follow this link to subscribe/unsubscribe:
> 	>>> http://www.cmake.org/mailman/listinfo/cmake
> 	>>>
> 	>>
> 	>>
> 
> 
> 
> 
> 
> This email was sent to you by Thomson Reuters, the global news and information company.
> Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters.
> 
> _______________________________________________
> Powered by www.kitware.com
> 
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
> 
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
> 
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake



More information about the CMake mailing list