[CMake] file(DOWNLOAD) and HTTP 302 (redirect)
David Cole
david.cole at kitware.com
Fri Jun 4 14:49:22 EDT 2010
Thanks, Michael and Aaron!
I pushed your patch (+ a SetError line) here in CMake 'next' :
http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=ef491f78218e255339278656bf6dc26073fef264
On Fri, Jun 4, 2010 at 11:38 AM, Michael Wild <themiwi at gmail.com> wrote:
> The following patch does work for me:
>
> diff --git a/Source/cmFileCommand.cxx b/Source/cmFileCommand.cxx
> index 5611527..3b18e0b 100644
> --- a/Source/cmFileCommand.cxx
> +++ b/Source/cmFileCommand.cxx
> @@ -2629,6 +2629,14 @@
> cmFileCommand::HandleDownloadCommand(std::vector<std::string>
> return false;
> }
>
> + res = ::curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
> + if (res != CURLE_OK)
> + {
> + std::string errstring = "FILE(DOWNLOAD ) error; cannot set
> follow-redirect option: ";
> + errstring += ::curl_easy_strerror(res);
> + return false;
> + }
> +
> if(verboseLog.size())
> {
> res = ::curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
> --
>
> However, I didn't test it thoroughly, and am not sure whether it would be a
> good idea to restrict the protocols and number of redirects.
>
> Michael
>
> On 4. Jun, 2010, at 17:08 , <aaron.meadows at thomsonreuters.com> <
> aaron.meadows at thomsonreuters.com> wrote:
>
> > I don't know what options are compiled in for CMake, but it looks like
> > this is the libcurl option required to follow redirects:
> >
> >
> >
> > CURLOPT_FOLLOWLOCATION
> >
> >
> >
> > A parameter set to 1 tells the library to follow any Location: header
> > that the server sends as part of an HTTP header.
> >
> >
> >
> > This means that the library will re-send the same request on the new
> > location and follow new Location: headers all the way until no more such
> > headers are returned. CURLOPT_MAXREDIRS can be used to limit the number
> > of redirects libcurl will follow.
> >
> >
> >
> > Since 7.19.4, libcurl can limit what protocols it will automatically
> > follow. The accepted protocols are set with CURLOPT_REDIR_PROTOCOLS and
> > it excludes the FILE protocol by default.
> >
> >
> >
> > Aaron C. Meadows
> >
> > ________________________________
> >
> > From: cmake-bounces at cmake.org [mailto:cmake-bounces at cmake.org] On Behalf
> > Of David Cole
> > Sent: Friday, June 04, 2010 9:47 AM
> > To: Michael Wild
> > Cc: CMake List
> > Subject: Re: [CMake] file(DOWNLOAD) and HTTP 302 (redirect)
> >
> >
> >
> > On Fri, Jun 4, 2010 at 8:03 AM, Michael Wild <themiwi at gmail.com> wrote:
> >
> > The log looks like this (I've put a message("log = ${log}") into
> > ExternalProject.cmake to get this, since the numeric error code is 0):
> >
> > <snip>
> > -- downloading...
> >
> > src='http://downloads.sourceforge.net/project/freefoam/ThirdParty/zlib/z
> > lib-1.2.5.tar.gz'
> >
> > dst='/Users/mwild/Projects/FreeFOAM/build-pu/ThirdParty/zlib/src/zlib-1.
> > 2.5.tar.gz'
> > timeout='none'
> > log = About to connect() to downloads.sourceforge.net port 80 (#0)
> > Trying 216.34.181.59... connected
> > Connected to downloads.sourceforge.net (216.34.181.59) port 80 (#0)
> > GET /project/freefoam/ThirdParty/zlib/zlib-1.2.5.tar.gz HTTP/1.1
> > Host: downloads.sourceforge.net
> > Accept: */*
> >
> > HTTP/1.1 302 Found
> > X-Powered-By: PHP/5.2.9
> > Content-Disposition: attachment; filename="zlib-1.2.5.tar.gz"
> > Location:
> > http://surfnet.dl.sourceforge.net/project/freefoam/ThirdParty/zlib/zlib-
> > 1.2.5.tar.gz
> > Content-type
> > <http://surfnet.dl.sourceforge.net/project/freefoam/ThirdParty/zlib/zlib
> > -1.2.5.tar.gz%0d%0aContent-type> : text/html
> > Content-Length: 0
> > Date: Fri, 04 Jun 2010 07:52:13 GMT
> > Server: lighttpd/1.4.26
> >
> > Connection #0 to host downloads.sourceforge.net left intact
> > Closing connection #0
> >
> > -- downloading... done
> > </snip>
> >
> > So you see, CMake really thinks this succeeded.
> >
> >
> >
> > Hmm. I wonder if there's some sort of curl option we should be passing
> > to get redirects to work automatically...?
> >
> >
> >
> >
> > I really am looking forward to the EXPECTED_MD5 feature, please
> > also make it available through ExternalProject_Add! Currently I'm doing
> > this manually via an additional step, but it's awfully fragile...
> >
> >
> >
> > It is. That was actually the main motivation for the additional
> > file(DOWNLOAD arg... In CMake 'next' there's a URL_MD5 arg to
> > ExternalProject_Add now.
> >
> >
> >
> >
> >
> >
> > Michael
> >
> >
> > On 4. Jun, 2010, at 13:57 , David Cole wrote:
> >
> > > I'm pretty sure the original intent was for file(DOWNLOAD ...)
> > to not raise
> > > errors so that scripts can analyze status and log and decide
> > what to do next
> > > rather than forcing a CMake error...
> > >
> > > I recently added EXPECTED_MD5 to file(DOWNLOAD in the 'next'
> > branch of
> > > CMake. If a download fails when there's an expected MD5 sum
> > given, then it
> > > *does* raise an error. Perhaps you'll find that useful in the
> > next
> > > release...
> > >
> > > Cheers,
> > > David C.
> > >
> > >
> > > On Fri, Jun 4, 2010 at 7:55 AM, David Cole
> > <david.cole at kitware.com> wrote:
> > >
> > >> What if you do something this...? What do status and log tell
> > you?
> > >>
> > >> set(remote "http://......")
> > >> set(local "some file name")
> > >>
> > >> file(DOWNLOAD
> > >>
> > >> "${remote}"
> > >>
> > >> "${local}"
> > >>
> > >> STATUS status
> > >>
> > >> LOG log)
> > >>
> > >>
> > >> list(GET status 0 status_code)
> > >>
> > >> list(GET status 1 status_string)
> > >>
> > >>
> > >> if(NOT status_code EQUAL 0)
> > >>
> > >> message(FATAL_ERROR "error: downloading '${remote}' failed
> > >>
> > >> status_code: ${status_code}
> > >>
> > >> status_string: ${status_string}
> > >>
> > >> log: ${log}
> > >>
> > >> ")
> > >>
> > >> endif()
> > >>
> > >>
> > >>
> > >> On Fri, Jun 4, 2010 at 4:00 AM, Michael Wild
> > <themiwi at gmail.com> wrote:
> > >>
> > >>> When downloading something from
> > http://downloads.sourceforge.net/. <http://downloads.sourceforge.net/>
> > .. you
> > >>> always get a 302 redirect to one of the many mirror servers.
> > Unfortunately,
> > >>> it seems that file(DOWNLOAD ...) doesn't follow the
> > redirect, but just stops
> > >>> and leaves an empty file behind. No error, no nothing. Is
> > this intentional?
> > >>>
> > >>> I'd rather not use a specific mirror...
> > >>>
> > >>> Michael
> > >>> _______________________________________________
> > >>> Powered by www.kitware.com
> > >>>
> > >>> Visit other Kitware open-source projects at
> > >>> http://www.kitware.com/opensource/opensource.html
> > >>>
> > >>> Please keep messages on-topic and check the CMake FAQ at:
> > >>> http://www.cmake.org/Wiki/CMake_FAQ
> > >>>
> > >>> Follow this link to subscribe/unsubscribe:
> > >>> http://www.cmake.org/mailman/listinfo/cmake
> > >>>
> > >>
> > >>
> >
> >
> >
> >
> >
> > This email was sent to you by Thomson Reuters, the global news and
> information company.
> > Any views expressed in this message are those of the individual sender,
> except where the sender specifically states them to be the views of Thomson
> Reuters.
> >
> > _______________________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.cmake.org/mailman/listinfo/cmake
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.cmake.org/pipermail/cmake/attachments/20100604/e2bc3ca7/attachment.htm>
More information about the CMake
mailing list