[cmake-developers] CMake daemon-mode

[Tobias Hunger]

Hi everybody,

I want the cmake daemon mode Stephen Kelly proposed a while back. In my opinion
this could become a game changer. Unfortunately Stephen told me that he did not
have the resources to push this forward at this time.

This is why I started working on making Stephen's code merge-able and more
robust. I will also volunteer to help maintain this code going forward.

My main interest is getting project structure reported out of cmake. So this is
what I did start out with.

The cool features Stephen demo-ed in his blog

https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/

are definitely on my wish list though, but I need first class cmake project
support first -- with all the information Qt Creator can currently not get its
hands on.

So I set up a branch over on github and did some hacking:

https://github.com/hunger/CMake/commits/daemon-mode



Status:
===============

So far I copied over the daemon itself from Stephen. On top of that I added some
nicer request/respond handling code (you get pretty similar messages from
everything, you can set a cookie in a request, which will be handed back on
response/error, there is a defined way to report a response or an error
condition), progress reporting (directly hooked up to cmake), and a couple of
basic commands to get started (reset, etc.), and some future-proofing by being
able to support several versions of the daemon protocol.

This infrastructure part is pretty solid by now, and has documentation and
tests.

On top of that I started to work towards loading a project and extracting the
project structure from it. This part is still pretty much in flux as I learn
where all the bits and pieces of data are stored in cmake.

This part is without proper documentation and tests at this time.

I do not consider this branch merge-worthy yet, but I would appreciate feedback
on the patch set *A LOT*, ranging from coding style issues up to hints about
things I should add to enable more use-cases, or hints on how to get to
information currently missing from the output.

A big chunk of Stephen's work has not even landed in my branch yet. Since cmake
reformated all the source in the meantime it is a bit tedious to apply patches
from his tree and I have simply not yet needed the changes as I did not venture
where he went yet. It also leaves daemon-mode as a rather separate set of
patches that is not changing core cmake components. I also hope this helps with
review at this time.



Run through:
===============

Start "cmake -E daemon" (no additional parameters required). This will enable
daemon-mode and allow you to send magic text via stdin to the daemon, who will
respond with magic text on stdout. We will probably want to change that later to
a channel that does not get interleaved with cmake output, but for now it works
well enough and is really easy to test.

A typical session will currently include the following messages sent to
the cmake daemon:

[== CMake MetaMagic ==[
{"protocolVersion":{"major":0,"minor":1},"type":"handshake"}
]== CMake MetaMagic ==]

This needs to be the first thing sent as it establishes the protocol version,
right after the daemon reports the available protocol versions on start-up.

[== CMake MetaMagic ==[
{"type":"setGlobalSettings", "currentGenerator":"Ninja",
"sourceDirectory":"/home/code/src/cmake", "buildDirectory":"/tmp/cmake-build-
test"}
]== CMake MetaMagic ==]

This sets up a new build directory /tmp/cmake-build-test for the sources found
in /home/code/src/cmake, using the Ninja generator. You should be able to leave
out the currentGenerator and the sourceDirectory if the buildDirectory already
exists.

[== CMake MetaMagic ==[
{"type":"configure"}
]== CMake MetaMagic ==]

Configures the project. You can also try:

[== CMake MetaMagic ==[
{"type":"configure", "cacheArguments":["-Dsomething=else"]}
]== CMake MetaMagic ==]

to pass configure arguments along to cmake.

Next step is:

[== CMake MetaMagic ==[
{"type":"generate"}
]== CMake MetaMagic ==]

which will generate the build system in the build directory. This and configure
will produce progress output along the way.

At this point you can query the project structure:

[== CMake MetaMagic ==[
{"type":"project"}
]== CMake MetaMagic ==]

This will dump a lot of output:-) You can trim down on that a bit by limiting
target types, etc.

Additional commands supported at this time:

[== CMake MetaMagic ==[
{"type":"globalSettings"}
]== CMake MetaMagic ==]

which lists the things you can set via "setGlobalSettings".

[== CMake MetaMagic ==[
{"type":"reset"}
]== CMake MetaMagic ==]

which will reset the daemon to its default state.



Todo:
===============

* Handle toolsets for generators

* Get CMakeCache via the daemon (after configure?)

* Get build system files from the daemon (after configure?)

* Get missing information on the project:
   * Defines
   * Compiler flags
   * Information on linkage of targets

* Group similar sources to shorten the output

* Support cmake --build via the daemon (optional?)

* Support cmake code completion

* Support cmake debugging



Help needed:
===============

Please help to support your use-cases.

Please report what information you would like to have available from your cmake
projects via daemon-mode.

Patches are of course welcome at any time!

Any help in getting any interesting information out of cmake is highly
appreciated, too.

At this time I think I will need to duplicate a chunk of code from one of the
generators to find the flags. Is that really necessary? If so: Which generator
should I copy the code from?

Best Regards,
Tobias

-- 
Tobias Hunger, Senior Software Engineer | The Qt Company
The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho. Sitz der
Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B