[cmake-developers] New command 'file(LOCK_DIRECTORY ...)'

[Ruslan Baratov]

On 18-Dec-14 01:49, Matthew Woehlke wrote:
> On 2014-12-17 16:58, Ruslan Baratov via cmake-developers wrote:
>> On 17-Dec-14 21:11, Matthew Woehlke wrote:
>>> On 2014-10-13 10:39, Brad King wrote:
>>>> On 10/10/2014 07:45 AM, Ruslan Baratov wrote:
>>>>> Locking directory is equivalent to locking file `cmake.lock` in
>>>>> directory "/path/to/shared/directory/":
>>>> I think this can be activated buy a DIRECTORY option.
>>> Why do we need even that? Can't CMake just test if the lock target is a
>>> directory?
>> I've used boost.interprocess as a hint for implementation, as far as I
>> know boost::interprocess::file_lock is not able to lock directories:
>>    terminate called after throwing an instance of
>> 'boost::interprocess::interprocess_exception'
>>      what():  Is a directory
>>
>> I'm not saying that it's not possible, just trying to use well-known
>> cross-platform solution.
> Locking a directory "as such" may not be possible (that wouldn't
> surprise me). That's not what I meant, though; what I meant is, rather
> than make the user say "this is a directory" to instead lock a special
> file name in the directory, just have CMake check if the path is a
> directory and then use that logic implicitly.
Okay, now I see what you mean :)
>   Requiring the user to
> specify seems superfluous when it's trivial for CMake to just check.
>
> Example:
>
>    file(LOCK /path/to/file) # lock /path/to/file
>    file(LOCK /path/to) # lock /path/to/.cmake_lock
`file(LOCK /path/to/file DIRECTORY)` and `file(LOCK /path/to/directory)` 
will report an error instead of silently lock creation.
> (And before anyone mentions it, I have limited sympathy for someone
> locking a non-existing directory and then later trying to create it.
> Don't do that :-). I'm having a very hard time coming up with a reason
> why one wouldn't just create the directory first to ensure it exists.)
Just for your information:
* there is no sub-option RECURSIVE for command `file(MAKE_DIRECTORY 
...)`, so all intermediate directories will be always created if they 
not exist.
* command `file(WRITE "/path/to/some/file.txt" "some info") will create 
all directories too
* command `install` create all directories. E.g. if some developer 
(well, bad developer) will hardcode '/usr/local' in 
CMAKE_INSTALL_PREFIX, binaries will be installed into `X:/usr/local` on 
windows.
>
> For that matter, do we even *need* this feature? If I want to lock a
> "directory", is it worth the extra implementation complexity just to
> save me inventing some arbitrary name for a lock file?

You prefer ".cmake.lock", I prefer "cmake.lock", others prefer 
'.lock'/'.cmake'. For me it doesn't really matter. What matter is the 
*standard* cmake name.

About "implementation complexity":

     @@ -3512,7 +3512,6 @@ bool cmFileCommand::HandleLockCommand(
     -  bool directory = false;
     @@ -3535,10 +3534,6 @@ bool cmFileCommand::HandleLockCommand(
     -    if (args[i] == "DIRECTORY")
     -      {
     -      directory = true;
     -      }
     @@ -3617,11 +3612,6 @@ bool cmFileCommand::HandleLockCommand(
     -  if (directory)
     -    {
     -    path += "/cmake.lock";
     -    }

>>> p.s. For a lock with no timeout, maybe there can be a built-in timeout
>>> after which CMake displays a message what it's trying to do, so that it
>>> doesn't just hang mysteriously.
>> You can use `message` command:
>>
>>      message(STATUS "Try to lock ${file}")
>>      file(LOCK ${file})
>>
>> or more complex (more user-friendly):
>>
>>    while(TRUE)
>>      file(LOCK "${file}" RESULT is_locked TIMEOUT 5)
>>      if(is_locked)
>>        message("Locked. ${resource_info}")
>>        break()
>>      endif()
>>      string(TIMESTAMP time_now "%S/%M")
>>      message("[${time_now}] Lock failed, retry... (file: ${file})")
>>    endwhile()
> Sure. I was just thinking out loud if folks are worried about CMake
> "mysteriously" hanging, this (implemented in CMake proper) would be an
> option to allay those concerns.
You can use `file(LOCK ...` as a base block for creation more 
complex/verbose locks (i.e. you can add CMakeLockHelpers extra module), 
I don't see any implementation defects that can't you allow to do one.

*Developers* do the "mysterious" stuff, not cmake. You can write:

   while(TRUE)
   endwhile()

so should we print extra message on entering `while`?

Though it would be the nice feature if you can read some info about 
lockers. E.g. `CMAKE_BINARY_DIR`, username, pid and timestamp. But this 
require a lot of work: lock new file for writing (cmake.lock-info), 
write some info (what format?), unlock (allow other process read info 
file) and so on.
>   (I'd also encourage anyone writing
> locks, especially with no time out, to always say what they're doing
> first :-). But that's somewhat a separate issue.)
>
I'm the one who will use this feature without timeouts (it's impossible 
to predict some). E.g. cmake instance will run ExternalProject_Add with 
Qt building for > 4 hours, should I set the timeout so my jenkins jobs 
will crash? Or should I print waiting message every second? What about > 
14400 lines of useless/endless logs? The important part is that if you 
abort the job, or "kill" the process, then lock will be released 
automagically. There are no instruments in cmake right now that can 
emulate such behaviour.

Ruslo