MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0015898CMakeCMakepublic2016-01-02 05:182016-06-10 14:31
ReporterIlya 
Assigned ToKitware Robot 
PrioritynormalSeverityfeatureReproducibilityN/A
StatusclosedResolutionmoved 
PlatformOSOS Version
Product VersionCMake 3.4.1 
Target VersionFixed in Version 
Summary0015898: CMake must provide a magic method to do right escaping of commands being executed in shell
DescriptionRight now it's very hard to write a command that should be executed in shell and will work on various platforms. Here is the notable example we came across recently: https://gist.github.com/Kentzo/00198d5cfc03ebdedddf [^]

CMake should provide a function or some way to specify a literal that will be correctly escaped according to the rules of the shell where the command will be executed.
Steps To Reproduce
Additional Informationhttps://gist.github.com/Kentzo/00198d5cfc03ebdedddf [^]
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2016-01-02 05:18IlyaNew Issue
2016-01-02 09:07Nils GladitzNote Added: 0040038
2016-01-03 03:03IlyaNote Added: 0040042
2016-01-03 04:57Nils GladitzNote Added: 0040043
2016-01-04 03:42IlyaNote Added: 0040047
2016-01-04 04:17Nils GladitzNote Added: 0040048
2016-01-05 08:08IlyaNote Added: 0040140
2016-01-05 09:28Nils GladitzNote Added: 0040141
2016-01-07 15:11Brad KingNote Added: 0040148
2016-01-25 02:39AntonioNote Added: 0040310
2016-01-25 04:48Nils GladitzNote Added: 0040311
2016-01-25 06:13AntonioNote Added: 0040312
2016-01-25 06:25Nils GladitzNote Added: 0040313
2016-01-25 08:43AntonioNote Added: 0040314
2016-01-25 08:53Nils GladitzNote Added: 0040317
2016-01-25 10:16AntonioNote Deleted: 0040314
2016-01-25 10:18AntonioNote Added: 0040322
2016-01-25 10:41AntonioNote Edited: 0040322bug_revision_view_page.php?bugnote_id=40322#r2019
2016-06-10 14:29Kitware RobotNote Added: 0042906
2016-06-10 14:29Kitware RobotStatusnew => resolved
2016-06-10 14:29Kitware RobotResolutionopen => moved
2016-06-10 14:29Kitware RobotAssigned To => Kitware Robot
2016-06-10 14:31Kitware RobotStatusresolved => closed

Notes
(0040038)
Nils Gladitz   
2016-01-02 09:07   
The add_custom_command() and add_custom_target() commands provide the VERBATIM option which is supposed to do this.
(0040042)
Ilya   
2016-01-03 03:03   
Some commands (such as ExternalProject) do not have it.

But doesn't it only covers escaping at the CMake level? I was thinking about a method that would handle *both* CMake escaping and shell escaping.
(0040043)
Nils Gladitz   
2016-01-03 04:57   
ExternalProject may or may not use a shell.
It may invoke the commands through add_custom_command() (with VERBATIM) or
it may generate a CMake script using execute_process() (which doesn't use a shell) and invoke that instead.

When using add_custom_command()/add_custom_target() directly VERBATIM is
supposed to take care of escaping required by the build tool.

I am not sure what you mean by CMake escaping.
Escaping required on the CMake script level is the same on all systems and is your responsibility as the caller and is processed before arguments are handed off to the commands themselves.
(0040047)
Ilya   
2016-01-04 03:42   
Nils, under what circumstances the COMMAND of ExternalProject may not use a shell?

There are two levels of escaping:
1. At CMake level to ensure a command will be passed to the underlying shell unmodified by CMake
2. At shell level to ensure it will be recognized as user intends it

The most notable problem which I think will be sufficient to address in 99% of the cases is escaping of quotes. A function I request should (at least) transform quotes into something that will not be handled at neither level 1 nor 2, like in the gist I attached.
(0040048)
Nils Gladitz   
2016-01-04 04:17   
ExternalProject does not run the command in a shell directly when you use one of the corresponding LOG options (e.g. LOG_BUILD for BUILD_COMMAND)

I don't follow your elaboration of escapes but escapes at the CMake script level have nothing to do with any shell.

E.g. quotation marks are interpreted by the CMake script language
as delimiters of quoted arguments.

So to pass literal quotes to any CMake command quotation marks have to be escaped.

You can reduce/eliminate the need for escapes at the CMake level by using the
bracket argument syntax that was introduced in 3.0.

The intention of VERBATIM is that CMake itself adds any quoting or escaping required so that the executed process sees the arguments as they are seen by the add_custom_command()/add_custom_target() commands (that is after CMake escapes have been processed).
(0040140)
Ilya   
2016-01-05 08:08   
Nils, do you mean with bracket argument syntax I do not need VERBATIM?

I.e. I can use something like:

    [=[sed /VERSION/s/build_number='.*'/build_number='${BUILD_NUMBER}'/ <SOURCE_DIR>/version.py.in > <SOURCE_DIR>/version.py]=]

Instead of:

    sed /VERSION/s/build_number=\'.*\'/build_number=\'${BUILD_NUMBER}\'/ <SOURCE_DIR>/version.py.in > <SOURCE_DIR>/version.py

?
(0040141)
Nils Gladitz   
2016-01-05 09:28   
No the bracket argument syntax helps you avoid CMake script level escaping.
With VERBATIM you avoid shell specific escapes.

Single quotes have no special meaning in CMake script so they do not have to be escaped for CMake's sake.
(0040148)
Brad King   
2016-01-07 15:11   
For reference, ExternalProject always passes VERBATIM to any add_custom_command call whose COMMAND can be specified by the caller:

 https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;hb=v3.4.1#l1531 [^]
(0040310)
Antonio   
2016-01-25 02:39   
Before filing a new feature request, I wanted to understand if there would be a way, when in VERBATIM mode (standard for execute_process), to pass a double quote (") into the argument line.

Please check this example:
http://stackoverflow.com/questions/34905194/cmake-how-to-call-execute-process-with-a-quote-in-the-argument [^]
(0040311)
Nils Gladitz   
2016-01-25 04:48   
To pass quotes directly to the command either [["foo"]] or \"foo\" should work.

I'm guessing the example you linked does not work since this is a windows binary
with a custom command line parser.

The CreateProcess API on windows does not handle individual arguments but rather
requires a single continuous string for the entire command.

CMake creates a string that matches the behaviour of the default parser.

E.g. with the default parser both the quoted argument "foo" and the unquoted argument foo result in the unquoted argument foo.

Hence if the windows "find" command were using the default parser it would not be able to tell the difference between quoted and unquoted arguments.
(0040312)
Antonio   
2016-01-25 06:13   
So, if I understand correctly, Windows's `find` cannot be used from within cmake, because of a limitation of the CreateProcess API and of the way `find` parses arguments.
(0040313)
Nils Gladitz   
2016-01-25 06:25   
You can not directly invoke window's find command from execute_process() but you could indirectly e.g.

  file(WRITE foo.bat [[find "searchterm" file.foo]])
  execute_process(COMMAND foo.bat)
(0040317)
Nils Gladitz   
2016-01-25 08:53   
Works for me as expected; note that the output of the command starts with a newline.
(0040322)
Antonio   
2016-01-25 10:18   
(edited on: 2016-01-25 10:41)
I have posted a working workaround for the `find` doublequoting problem here: http://stackoverflow.com/a/34995590/2436175 [^]
(0042906)
Kitware Robot   
2016-06-10 14:29   
Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.