[CMake] CMake on host executing compiler inside docker

[Federico Kircheis]

Hello,

I'm trying to solve following problem.

test as many different compilers as possible with no (or minimal) 
maintenance burden.

As docker is the newest trend, I decided to give it a try (nearly using 
it for the first time).
Also the GCC team has official docker images 
(https://hub.docker.com/_/gcc/), so I decided to give it a try.

The main advantage is that I do not need to manage, compile and package 
myself all different compiler versions for my operating system.


The issue is that the integration between tools inside docker images and 
tools outside docker images is very bad.


I do not want to install CMake (and other tools) on those images as I 
could see on many projects because

   * it does not scale at all if I want to also test different version 
of CMake, make, ninja, clang, ...
   * It's a maintenance burden, as every time the base image gets 
updated, I need to update my modified images too
   * If I modify the official image, I need to know internal details 
that are not really relevant for testing the compiler (what OS, package 
manager, directory structure, ...)
   * It does not solve the integration issue, I do not think that 
installing the whole IDE would be a good idea.


At that point, compiling and packaging for the host platform is probably 
easier.


So what I tried was to take the docker images, and let my local CMake 
(CMake on the host, not in docker), use the compiler in the docker image.

This would have following advantages

   * compiler and build system are decoupled, I can easily test many 
version combinations of the two
   * No need to know how the image is build, I'm using it as a black 
box, as originally intended.
   * As most IDE understand make or CMake, I have the integration with 
the dockerized compiler for free (probably for other tools too)

I've created following toolchain file

----
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_CROSSCOMPILING TRUE)

# FIXME: instead of hard coding 1000, map for real user id...
# Good enough for now as generated files are not owned by root
set(command 
"docker;run;--env;VERBOSE=1;-v;${CMAKE_SOURCE_DIR}:${CMAKE_SOURCE_DIR};-w;${CMAKE_SOURCE_DIR};--user;1000;gcc:5")

foreach(LANG C CXX ASM)
	set(CMAKE_${LANG}_COMPILER_LAUNCHER ${command} CACHE STRING "")
endforeach()
#set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "docker run --env 
VERBOSE=1 -v ${CMAKE_SOURCE_DIR}:${CMAKE_SOURCE_DIR} -w 
${CMAKE_SOURCE_DIR} --user 1000 gcc:5")


set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
----

(the second attempt was setting `set_property(GLOBAL PROPERTY 
RULE_LAUNCH_COMPILE "docker run --env VERBOSE=1 -v 
${CMAKE_SOURCE_DIR}:${CMAKE_SOURCE_DIR} -w ${CMAKE_SOURCE_DIR} --user 
1000 gcc:5")`, but it did not make any difference)

and used it like

----

m -rf build;cmake -S . -B build 
-DCMAKE_TOOLCHAIN_FILE=toolchain-docker.cmake --debug-trycompile && 
cmake --build build
----

I can see that the makefiles all have "docker run ...." commands, which 
looks great, but CMake, in it's internal logic, does not seem to always 
apply the `RULE_LAUNCH_COMPILE` and `CMAKE_${LANG}_COMPILER_LAUNCHER`, 
as the output looks like:

----
-- The C compiler identification is GNU 9.2.1 
 

-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
----

GCC 9.2.1 is the compiler on my local host, the output should have been 
version 5.

I've also noticed that internal tests, like 
/usr/share/cmake-3.13/Modules/CMakeTestCCompiler.cmake:52, gets executed 
with the "docker run" prefix, and they do not fail.


Is there some parameter that I'm missing, or is there a better way to 
achieve what I'm trying to do?

I can also verify that CMake does not always apply "RULE_LAUNCH_COMPILE" 
and "CMAKE_${LANG}_COMPILER_LAUNCHER", because if I add 
"set(CMAKE_C_COMPILER /usr/local/bin/gcc)" (which is the internal path 
in docker), then CMake prints

----
-- The C compiler identification is unknown
-- The ASM compiler identification is unknown
-- Didn't find assembler
CMake Error at CMakeLists.txt:12 (project):
   The CMAKE_C_COMPILER:

     /usr/local/bin/gcc

   is not a full path to an existing compiler tool.

   Tell CMake where to find the compiler by setting either the environment
   variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full 
path to
   the compiler, or to the compiler name if it is in the PATH.
----

as GCC on my host is "/usr/bin/gcc".