[cmake-developers] New command 'file(LOCK_DIRECTORY ...)'

[Ruslan Baratov]

On 10-Oct-14 16:58, David Cole wrote:
> What is the main use case for locking files and directories from the
> CMake language?
>
> I feel slightly uncomfortable, without being able to put my finger on
> exactly why, with the prospect of adding this to CMake...
I've described the situation in the first message of this thread: 
sharing resources between different cmake builds. If you want the 
particular case: I'm developing cmake-based package manager that share 
the builds of ExternalProject_Add. I'm experienced problems even when 
working on my local machine (I've started long-build in one console, 
then accidentally connect to the same shared directory in other 
console), but the main problem here is the server auto-builds which 
usually start multiple jobs simultaneously.
>
> Behavior when there's a lock that still exists (by bug/mistake) after
> CMake is done running is going to be strange and possibly
> hard-to-diagnose / hard-to-understand.
That's *exactly* the problem I have and why I start this discussion. 
Currently I'm using mkdir command which return 0 only if directory 
created by current process (just to clarify, not the cmake -E 
make_directory). This works fine for me (tested on windows (mingw, 
cygwin), linux and mac). But I can't (or I don't know how) make remove 
this directory when I Ctrl+C my build (or abort the job on jenkins 
server). My current implementation just keep spinning with message: 
"Directory locked by cmake, build started in directory 
<cmake-binary-dir> at <timestamp>".
>
> Let's make sure the use case is quite strong before we accept the
> possibility of "stale locks" in CMake.
Command `file(LOCK_GUARD ...)` will cover 100% of my needs. Other 
commands that I've mentioned used to cover suggestions from Brad K. (may 
be I've just understand them wrong).

Ruslo

Thanks, David C. On Fri, Oct 10, 2014 at 7:45 AM, Ruslan Baratov via 
cmake-developers <cmake-developers at cmake.org> wrote:
>> On 08-Oct-14 16:45, Brad King wrote:
>>> On 10/08/2014 07:52 AM, Ruslan Baratov wrote:
>>>> Okay :) I just not sure that I understand "to pass to some other
>>>> process" goal correctly.
>>> That was just an example of why one might want to drop management
>>> of the lock by CMake without actually unlocking it.  Perhaps the
>>> code is starting a daemon and passes off responsibility for the
>>> unlock side to the daemon process.
>> Okay, got it.
>>>
>>>> * we just need to `unlock` file so the other instance will use it:
>>>> file(UNLOCK_FILE "/path/to/shared/file")
>>>> # now anybody can do the lock
>>> Yes.  We also need the locking API to return information about
>>> whether we really acquired the lock.
>> So it can be TRY_LOCK and LOCK. But does the TRY_LOCK really needed? With
>> LOCK we can try to lock and spin until acquire, but what to do with
>> TRY_LOCK? Just give up?
>>>
>>>> * we need other process to "inherit" the lock (what for?), i.e. move
>>>> ownership (?):
>>>> file(LOCK_FILE "/path/to/shared/file")
>>>> execute_process(${CMAKE_COMMAND} --take-ownership "/path/to/shared/file"
>>>> ...)
>>>> # unlocked by execute_process
>>> I think all we need there is a way to ask CMake to take over
>>> responsibility for a lock that it did not originally create.
>>> It can also be in the file() command.
>>>
>>> Thanks,
>>> -Brad
>>>
>> Okay, so here is the commands inspired by C++:
>>
>>    file(TRY_LOCK "/path/to/file" result) # try to lock and return TRUE on
>> success (needed?)
>>    file(LOCK ...) # lock until unlock (unlock by further code (what if
>> exit/crashed?) or by deamon)
>>    file(LOCK_GUARD ...) # lock for scope of current function or until exit
>>    file(UNLOCK ...) # explicit unlock of LOCK/LOCK_GUARD
>>
>> Locking directory is equivalent to locking file `cmake.lock` in directory
>> "/path/to/shared/directory/":
>>
>>    file(DIRECTORY_TRY_LOCK ...)
>>    file(DIRECTORY_LOCK "/path/to/shared/directory/") # == file(LOCK
>> "/path/to/shared/directory/cmake.lock")
>>    file(DIRECTORY_LOCK_GUARD ...)
>>    file(DIRECTORY_UNLOCK ...)
>>
>> If any of this commands failed (e.g. have no permissions) - exit with
>> unsuccessful code (i.e. FATAL_ERROR) ?
>>
>> * http://en.cppreference.com/w/cpp/thread/try_lock
>> * http://en.cppreference.com/w/cpp/thread/lock
>> * http://en.cppreference.com/w/cpp/thread/lock_guard
>> * http://en.cppreference.com/w/cpp/thread/mutex/unlock
>>
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake-developers