[CMake] ExternalProject, continuous integration and caching builds

[Craig Scott]

On Wed, Sep 5, 2018 at 9:56 PM, Antoine Pitrou <antoine at python.org> wrote:

>
> Hello,
>
> On our project (Apache Arrow - https://arrow.apache.org/) we're using
> CMake for the C++ source tree and have many external dependencies
> fetched using ExternalProject.  In turn building those dependencies can
> make up a significant portion of build times on CI services, especially
> AppVeyor.  So I've been looking for a solution to cache those
> third-party builds from one CI run to the other.
>
> Right now, what I'm trying to do is to set EP_BASE to a well-known base
> directory and ask AppVeyor to cache and restore that directory in each
> new build.  The AppVeyor caching seems to work fine (the EP_BASE
> directory is saved and restored).  However, it seems that nevertheless
> CMake will rebuild all those projects again, despite the cached build
> results.
>

When AppVeyor restores the cached directories and files, does it also
preserve their timestamps? If not, that might explain why it always
rebuilds.



>
> This is with CMake 3.12.1 on Windows.
>
> Here is the log for an example build step, here the zstd library:
> https://ci.appveyor.com/project/pitrou/arrow/build/1.
> 0.700/job/i4tj6tifp4xq1mjn?fullLog=true#L803
>
> As you can see, CMake notices the downloaded tarball is up-to-date and
> doesn't download it again, but it still extracts it again (why?) and
> builds the source code anew.  Yet the entire EP_BASE directory (here
> "C:/Users/appveyor/arrow-externals") is cached and restored by AppVeyor.
>
> Did someone manage to make this work, and/or is there another solution?
>
>

-- 
Craig Scott
Melbourne, Australia
https://crascit.com

New book released: Professional CMake: A Practical Guide
<https://crascit.com/professional-cmake/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cmake.org/pipermail/cmake/attachments/20180905/991506a8/attachment.html>