[cmake-developers] ExternalProject and git clone

[Daniele E. Domenichelli]

On 09/09/13 22:17, David Cole wrote:
> ExternalProject makes some simplifying assumptions: it assumes the >
project is *External* to your own and that you won't be locally
> modifying it independent of the commands that it executes. In your
> case, that's a bad assumption.

Are you suggesting to add a new "SubProject" module instead? It would be
quite similar to ExternalProject though, but perhaps it makes sense...

Also searching for other "superbuilds" that use ExternalProject for
"internal" projects, I found that KDE superbuild[1] has a variable
SB_SRC_CLONE_DIR that is documented as "If you're frequently working on
the sources, then you'll want to have all of them inside a separate
directory", see [2]. Again this behaviour is not documented.
I'm not sure if KDE superbuild is currently used/developed or just
abandoned (last commit is 2 years old), but I believe that I'm not the
only one trying to use it in this way, and it should be a common concern
that the source directory is deleted when you delete the build directory...


[1]https://projects.kde.org/projects/kde/superbuild
[2]https://projects.kde.org/projects/kde/superbuild/repository/revisions/3ec2909821e408fad66d634f21fa771d5d775dd9/diff/SuperBuild.cmake


> On Sep 5, 2013, at 8:28 AM, Daniele E. Domenichelli wrote:
>> I would like to try to write a patch that modifies this behaviour
>> somehow, for example:
>>
>> * Skip the clone phase if the source_dir exists and is not in the build
>> directory
>> * Have a bool property to skip the clone phase if the directory exists
>> * Just skip the clone if the directory exists
>>
>> Does it seem reasonable? Do you think that this could cause other issues?
> 
> You can't just skip the clone unconditionally if the directory
> exists, because it may be pointing to a stale repository. The
> present code, (full rebuilds not counting), attempts to minimize
> re-cloning to the case where the URL to the git repo changes. But
> if the URL does change, we need to blow away the existing one and
> do a full re-clone... Even if the directory is there to start with.

This could be checked comparing the new url with
 git config --get remote.origin.url

And even if the url is different, it could still rename or delete the
origin remote, add the new url as origin, and fetch and checkout from
the new url. In this way, local changes wouldn't be lost.

But, of course, this works for git, I don't know if you can do it for
the other supported systems.

Anyway, just renaming the directory instead of removing it could be an
almost acceptable behaviour (at least my changes would not be lost forever)

> You could do any of those behaviors you ask about, they all seem
> reasonable, but they have to be optional. To keep existing projects
> working (and users happy!), CMake should have the same default
> behavior as it has now. And the new behavior would have to be
> explicitly switched on by a new parameter to ExternalProject_Add.
> 
> Additionally, you could patch CMake to have the missing
> documentation you mention above.
> 
> Let us know if you have further questions about this. Feel free to
> send a proposed patch along if you do create one.

Then I think I'll go for the first option ("Skip the clone phase if the
source_dir exists and is not in the build directory"), perhaps adding a
variable to enable/disable this, add some checks, and probably add some
documentation. I'll let you know when I have a patch ready.

Thanks!


Cheers,
 Daniele