[cmake-developers] ExternalProject and git clone

[David Cole]

On Sep 5, 2013, at 8:28 AM, Daniele E. Domenichelli wrote:

> ...
> I'd like to be able to modify files, commit, and push from the external
> git repositories, therefore I'm downloading them in a different
> directory from the build directory. Unfortunately if I want to rebuild
> from the start and I delete the build directory, the repository is
> deleted because the time stamps are missing, and then cloned again next
> time I run make
> ...
> 
> Therefore all my changes, branches, additional remotes, etc would be
> lost. Also no warning is given to the user that the directory is being
> deleted, and this behaviour doesn't seem to be documented anywhere.
> ...
> 

ExternalProject makes some simplifying assumptions: it assumes the project is *External* to your own and that you won't be locally modifying it independent of the commands that it executes. In your case, that's a bad assumption.

The primary use cases we considered when developing it were building snapshots of projects at known commits, and tracking an upstream repository branch for automatically and incrementally building the "latest" upstream commits.

However, you can do what you want, perhaps as simply as having your own custom download and update commands. Rather than specifying the GIT_REPOSITORY and other GIT_* params to ExternalProject_Add, you could simply use:

    DOWNLOAD_COMMAND git clone blahblahblah.git
    UPDATE_COMMAND git pull

... or something to that effect. Or, instead of running git commands directly, run scripts that verify conditions as you would like them to be.



> I would like to try to write a patch that modifies this behaviour
> somehow, for example:
> 
> * Skip the clone phase if the source_dir exists and is not in the build
> directory
> * Have a bool property to skip the clone phase if the directory exists
> * Just skip the clone if the directory exists
> 
> Does it seem reasonable? Do you think that this could cause other issues?


You can't just skip the clone unconditionally if the directory exists, because it may be pointing to a stale repository. The present code, (full rebuilds not counting), attempts to minimize re-cloning to the case where the URL to the git repo changes. But if the URL does change, we need to blow away the existing one and do a full re-clone... Even if the directory is there to start with.

You could do any of those behaviors you ask about, they all seem reasonable, but they have to be optional. To keep existing projects working (and users happy!), CMake should have the same default behavior as it has now. And the new behavior would have to be explicitly switched on by a new parameter to ExternalProject_Add.

Additionally, you could patch CMake to have the missing documentation you mention above.

Let us know if you have further questions about this. Feel free to send a proposed patch along if you do create one.


HTH,
David C.