[cmake-developers] Setting up environment using ExternalProject_Add

[James Johnston]

> -----Original Message-----
> From: Brad King [mailto:brad.king at kitware.com]
> Sent: Tuesday, August 11, 2015 13:35
> To: James Johnston
> Cc: cmake-developers at cmake.org
> Subject: Re: [cmake-developers] Setting up environment using
> ExternalProject_Add
> 
> On 08/11/2015 12:49 AM, James Johnston wrote:
> >         CONFIGURE_ENVIRONMENT_COMMAND <path to VCVarsAll.bat>
> [snip]
> >     add_custom_command(<snip> ENVIRONMENT_COMMAND <path to
> vcvarsall.bat>
> >         COMMAND ${CMAKE_PROGRAM} -GNinja <snip>)
> 
> A problem with both of these approaches is that on UNIX and in Make the
> environment does not persist after the command exits.

Exactly right, I'm aware... and also it seems there is no
good/easy/non-hackish way to capture the environment from a child process
after it terminates, either - that I'm aware of.  (It would be nice if we
could run a process, then capture its environment after it exits, but I
don't think it can be done.)

> Whatever wrapper
> tool is used must set the environment and also launch the real command to
> be run under that environment.  This is what the "env" tool (and recently
> "cmake -E env") does.

Right, but the problem with cmake -E env is it assumes you know what
variables you want to set - we don't.  Only a shell script / batch file
knows in the case of VCVarsAll.bat...

I think I mentioned this solution in the original e-mail but in more detail,
what I'm proposing with the proposed "cmake -E run_commands" or whatever we
want to call it would be the following.  In fact upon further thought, it
might be better to see about enhancing the "cmake -E env" command but I am
not sure if this is possible due to the syntax of "cmake -E env".  Anyhow
the basic idea is this: 
	
1.  CMake creates a temporary shell script / batch file (the shell chosen
based on the platform CMake compiled for - cmd.exe on Windows, sh on POSIX).
2.  CMake calls one or more "environment setup" commands.  These are pulled
in using call / source.  Users would be required to provide scripts for
environment setup in the format of the native shell (e.g. you can't "call" a
bash script from a Windows .bat file).  For example:

    @echo off
    rem User script provided to CMake, including some arguments to the
script
    call <dir>\VCVarsAll.bat x86
    rem An additional environment variable explicitly specified by the user
    set MyVar=Value
    cmake -GNinja .

Or:

    #!/bin/sh
    source setMyEnv.sh
    cmake -GNinja .

Error handling has been cut for brevity, but we would want the script to
check for ERRORLEVEL / $? after each line above.  As you can see, the
problem of environment not being retained has been solved by using "call"
and "source".

3.  Then CMake runs the above script as normal and checks the exit code as
normal.

It's basically a close cousin of what "cmake -E env" does in that we: (1) do
some things to set up the environment, (2) run the specified EXE.  Ideally
cmake -E env could be enhanced with the new functionality, but since we now
want to pass an arbitrary list of commands, each with potentially unlimited
parameters of any format, I'm not sure how that should look in practice.  It
might be easier to make a new command "cmake -E env2" or whatever (need to
find a better name):
  * It could be passed a file.  The file contents would have a series of
lines, either "SET / UNSET name=[value]", "SOURCE VCVarsAll.bat x86", or
"COMMAND cmake.exe <snip>".  Newlines delimit commands.
  * It could be passed all the commands directly on the command line.  A
recognized "--" switch delimits commands.  E.g. cmake -E env2 --unset NAME
--set NAME=VALUE --source VCVarsAll.bat x86 --command cmake.exe <snip>
--source moreSetup.bat --command somethingelse.exe
Downsides with first method: file adds complexity.  Downsides with second
method:  subject to command line length limits, though probably not an issue
in practice.  Also, what if user wants to pass "--unset" or "--set"
parameter to their command as an argument?  Without a means of escaping,
CMake will think the command is ended and the user is trying to do something
new.
	
That's the first and most important step.  The above functionality would
still be inconvenient to use (e.g. "cmake -E" will not be aware of any
compiled EXE target output paths).  So, enhancing e.g. add_custom_command as
I proposed might make it easier, as this command could be aware of compiled
EXE targets.  Finally, we'd want to add features to ExternalProject that use
the new CMake functionality to finally solve the original problem.

The alternative ideas in my e-mail are just a question of "who" generates
the above temporary script that "sources/calls" a user script.  E.g. in
option 3, we leave C++ portion of CMake alone, and ExternalProject would
then have to auto-generate the above temporary script instead.

Best regards,

James Johnston