(0040038)
Nils Gladitz (developer)
2016-01-02 09:07

The add_custom_command() and add_custom_target() commands provide the VERBATIM option which is supposed to do this.

(0040042)
Ilya (reporter)
2016-01-03 03:03

Some commands (such as ExternalProject) do not have it.

But doesn't it only covers escaping at the CMake level? I was thinking about a method that would handle *both* CMake escaping and shell escaping.

(0040043)
Nils Gladitz (developer)
2016-01-03 04:57

ExternalProject may or may not use a shell.
It may invoke the commands through add_custom_command() (with VERBATIM) or
it may generate a CMake script using execute_process() (which doesn't use a shell) and invoke that instead.

When using add_custom_command()/add_custom_target() directly VERBATIM is
supposed to take care of escaping required by the build tool.

I am not sure what you mean by CMake escaping.
Escaping required on the CMake script level is the same on all systems and is your responsibility as the caller and is processed before arguments are handed off to the commands themselves.

(0040047)
Ilya (reporter)
2016-01-04 03:42

Nils, under what circumstances the COMMAND of ExternalProject may not use a shell?

There are two levels of escaping:
1. At CMake level to ensure a command will be passed to the underlying shell unmodified by CMake
2. At shell level to ensure it will be recognized as user intends it

The most notable problem which I think will be sufficient to address in 99% of the cases is escaping of quotes. A function I request should (at least) transform quotes into something that will not be handled at neither level 1 nor 2, like in the gist I attached.

(0040048)
Nils Gladitz (developer)
2016-01-04 04:17

ExternalProject does not run the command in a shell directly when you use one of the corresponding LOG options (e.g. LOG_BUILD for BUILD_COMMAND)

I don't follow your elaboration of escapes but escapes at the CMake script level have nothing to do with any shell.

E.g. quotation marks are interpreted by the CMake script language
as delimiters of quoted arguments.

So to pass literal quotes to any CMake command quotation marks have to be escaped.

You can reduce/eliminate the need for escapes at the CMake level by using the
bracket argument syntax that was introduced in 3.0.

The intention of VERBATIM is that CMake itself adds any quoting or escaping required so that the executed process sees the arguments as they are seen by the add_custom_command()/add_custom_target() commands (that is after CMake escapes have been processed).

(0040140)
Ilya (reporter)
2016-01-05 08:08

Nils, do you mean with bracket argument syntax I do not need VERBATIM?

I.e. I can use something like:

[=[sed /VERSION/s/build_number='.*'/build_number='${BUILD_NUMBER}'/ <SOURCE_DIR>/version.py.in > <SOURCE_DIR>/version.py]=]

Instead of:

sed /VERSION/s/build_number=\'.*\'/build_number=\'${BUILD_NUMBER}\'/ <SOURCE_DIR>/version.py.in > <SOURCE_DIR>/version.py

?

(0040141)
Nils Gladitz (developer)
2016-01-05 09:28

No the bracket argument syntax helps you avoid CMake script level escaping.
With VERBATIM you avoid shell specific escapes.

Single quotes have no special meaning in CMake script so they do not have to be escaped for CMake's sake.

(0040148)
Brad King (manager)
2016-01-07 15:11

For reference, ExternalProject always passes VERBATIM to any add_custom_command call whose COMMAND can be specified by the caller:

https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;hb=v3.4.1#l1531 [^]

(0040310)
Antonio (reporter)
2016-01-25 02:39

Before filing a new feature request, I wanted to understand if there would be a way, when in VERBATIM mode (standard for execute_process), to pass a double quote (") into the argument line.

Please check this example:
http://stackoverflow.com/questions/34905194/cmake-how-to-call-execute-process-with-a-quote-in-the-argument [^]

(0040311)
Nils Gladitz (developer)
2016-01-25 04:48

To pass quotes directly to the command either [["foo"]] or \"foo\" should work.

I'm guessing the example you linked does not work since this is a windows binary
with a custom command line parser.

The CreateProcess API on windows does not handle individual arguments but rather
requires a single continuous string for the entire command.

CMake creates a string that matches the behaviour of the default parser.

E.g. with the default parser both the quoted argument "foo" and the unquoted argument foo result in the unquoted argument foo.

Hence if the windows "find" command were using the default parser it would not be able to tell the difference between quoted and unquoted arguments.

(0040312)
Antonio (reporter)
2016-01-25 06:13

So, if I understand correctly, Windows's `find` cannot be used from within cmake, because of a limitation of the CreateProcess API and of the way `find` parses arguments.

(0040313)
Nils Gladitz (developer)
2016-01-25 06:25

You can not directly invoke window's find command from execute_process() but you could indirectly e.g.

file(WRITE foo.bat [[find "searchterm" file.foo]])
execute_process(COMMAND foo.bat)

(0040317)
Nils Gladitz (developer)
2016-01-25 08:53

Works for me as expected; note that the output of the command starts with a newline.

(0040322)
Antonio (reporter)
2016-01-25 10:18
edited on: 2016-01-25 10:41

I have posted a working workaround for the `find` doublequoting problem here: http://stackoverflow.com/a/34995590/2436175 [^]

(0042906)
Kitware Robot (administrator)
2016-06-10 14:29

Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

Notes
(0040038) Nils Gladitz (developer) 2016-01-02 09:07	The add_custom_command() and add_custom_target() commands provide the VERBATIM option which is supposed to do this.

(0040042) Ilya (reporter) 2016-01-03 03:03	Some commands (such as ExternalProject) do not have it. But doesn't it only covers escaping at the CMake level? I was thinking about a method that would handle both CMake escaping and shell escaping.

(0040043) Nils Gladitz (developer) 2016-01-03 04:57	ExternalProject may or may not use a shell. It may invoke the commands through add_custom_command() (with VERBATIM) or it may generate a CMake script using execute_process() (which doesn't use a shell) and invoke that instead. When using add_custom_command()/add_custom_target() directly VERBATIM is supposed to take care of escaping required by the build tool. I am not sure what you mean by CMake escaping. Escaping required on the CMake script level is the same on all systems and is your responsibility as the caller and is processed before arguments are handed off to the commands themselves.

(0040047) Ilya (reporter) 2016-01-04 03:42	Nils, under what circumstances the COMMAND of ExternalProject may not use a shell? There are two levels of escaping: 1. At CMake level to ensure a command will be passed to the underlying shell unmodified by CMake 2. At shell level to ensure it will be recognized as user intends it The most notable problem which I think will be sufficient to address in 99% of the cases is escaping of quotes. A function I request should (at least) transform quotes into something that will not be handled at neither level 1 nor 2, like in the gist I attached.

(0040048) Nils Gladitz (developer) 2016-01-04 04:17	ExternalProject does not run the command in a shell directly when you use one of the corresponding LOG options (e.g. LOG_BUILD for BUILD_COMMAND) I don't follow your elaboration of escapes but escapes at the CMake script level have nothing to do with any shell. E.g. quotation marks are interpreted by the CMake script language as delimiters of quoted arguments. So to pass literal quotes to any CMake command quotation marks have to be escaped. You can reduce/eliminate the need for escapes at the CMake level by using the bracket argument syntax that was introduced in 3.0. The intention of VERBATIM is that CMake itself adds any quoting or escaping required so that the executed process sees the arguments as they are seen by the add_custom_command()/add_custom_target() commands (that is after CMake escapes have been processed).

(0040140) Ilya (reporter) 2016-01-05 08:08	Nils, do you mean with bracket argument syntax I do not need VERBATIM? I.e. I can use something like: [=[sed /VERSION/s/build_number='.'/build_number='${BUILD_NUMBER}'/ <SOURCE_DIR>/version.py.in > <SOURCE_DIR>/version.py]=] Instead of: sed /VERSION/s/build_number=\'.\'/build_number=\'${BUILD_NUMBER}\'/ <SOURCE_DIR>/version.py.in > <SOURCE_DIR>/version.py ?

(0040141) Nils Gladitz (developer) 2016-01-05 09:28	No the bracket argument syntax helps you avoid CMake script level escaping. With VERBATIM you avoid shell specific escapes. Single quotes have no special meaning in CMake script so they do not have to be escaped for CMake's sake.

(0040148) Brad King (manager) 2016-01-07 15:11	For reference, ExternalProject always passes VERBATIM to any add_custom_command call whose COMMAND can be specified by the caller: https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;hb=v3.4.1#l1531 [^]

(0040310) Antonio (reporter) 2016-01-25 02:39	Before filing a new feature request, I wanted to understand if there would be a way, when in VERBATIM mode (standard for execute_process), to pass a double quote (") into the argument line. Please check this example: http://stackoverflow.com/questions/34905194/cmake-how-to-call-execute-process-with-a-quote-in-the-argument [^]

(0040311) Nils Gladitz (developer) 2016-01-25 04:48	To pass quotes directly to the command either [["foo"]] or \"foo\" should work. I'm guessing the example you linked does not work since this is a windows binary with a custom command line parser. The CreateProcess API on windows does not handle individual arguments but rather requires a single continuous string for the entire command. CMake creates a string that matches the behaviour of the default parser. E.g. with the default parser both the quoted argument "foo" and the unquoted argument foo result in the unquoted argument foo. Hence if the windows "find" command were using the default parser it would not be able to tell the difference between quoted and unquoted arguments.

(0040312) Antonio (reporter) 2016-01-25 06:13	So, if I understand correctly, Windows's `find` cannot be used from within cmake, because of a limitation of the CreateProcess API and of the way `find` parses arguments.

(0040313) Nils Gladitz (developer) 2016-01-25 06:25	You can not directly invoke window's find command from execute_process() but you could indirectly e.g. file(WRITE foo.bat [[find "searchterm" file.foo]]) execute_process(COMMAND foo.bat)

(0040317) Nils Gladitz (developer) 2016-01-25 08:53	Works for me as expected; note that the output of the command starts with a newline.

(0040322) Antonio (reporter) 2016-01-25 10:18 edited on: 2016-01-25 10:41	I have posted a working workaround for the `find` doublequoting problem here: http://stackoverflow.com/a/34995590/2436175 [^]

(0042906) Kitware Robot (administrator) 2016-06-10 14:29	Resolving issue as `moved`. This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

Issue History
Date Modified	Username	Field	Change
2016-01-02 05:18	Ilya	New Issue
2016-01-02 09:07	Nils Gladitz	Note Added: 0040038
2016-01-03 03:03	Ilya	Note Added: 0040042
2016-01-03 04:57	Nils Gladitz	Note Added: 0040043
2016-01-04 03:42	Ilya	Note Added: 0040047
2016-01-04 04:17	Nils Gladitz	Note Added: 0040048
2016-01-05 08:08	Ilya	Note Added: 0040140
2016-01-05 09:28	Nils Gladitz	Note Added: 0040141
2016-01-07 15:11	Brad King	Note Added: 0040148
2016-01-25 02:39	Antonio	Note Added: 0040310
2016-01-25 04:48	Nils Gladitz	Note Added: 0040311
2016-01-25 06:13	Antonio	Note Added: 0040312
2016-01-25 06:25	Nils Gladitz	Note Added: 0040313
2016-01-25 08:43	Antonio	Note Added: 0040314
2016-01-25 08:53	Nils Gladitz	Note Added: 0040317
2016-01-25 10:16	Antonio	Note Deleted: 0040314
2016-01-25 10:18	Antonio	Note Added: 0040322
2016-01-25 10:41	Antonio	Note Edited: 0040322
2016-06-10 14:29	Kitware Robot	Note Added: 0042906
2016-06-10 14:29	Kitware Robot	Status	new => resolved
2016-06-10 14:29	Kitware Robot	Resolution	open => moved
2016-06-10 14:29	Kitware Robot	Assigned To	=> Kitware Robot
2016-06-10 14:31	Kitware Robot	Status	resolved => closed

Relationships