[cmake-developers] CMake alternative language

[Brad King]

Hi Folks,

I'm replying directly to my previous post in this thread in order to consolidate
responses to related discussion raised in others' responses to it:

 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15339/focus=15383
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15339/focus=15386
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15339/focus=15389

General comments:

* There is a lot of code out there in the current CMake language so I do not
  think it is realistic to drop it.  I'm not proposing that this change.

* CMake's procedural/imperative design is good as the main entry point to
  configuration of a project.  It can do system introspection, file generation,
  etc.  I'm not proposing that this change.

* Many projects build elaborate macro/function systems in the CMake language
  in order to end up with a declarative specification listing the actual
  source files, dependencies, and usage requirements.  I'd like to offer
  an alternative to this.

* Integration with IDEs is currently based on one-way generation (VS IDE
  projects, Xcode projects, CodeBlocks, etc.).  Editing the project build
  specification requires editing CMake code directly because IDEs cannot
  easily pierce CMake's procedural/imperative specification:

    https://cmake.org/pipermail/cmake-developers/2016-January/027386.html

  I'd like to improve this by *optionally* moving part of the specification
  to a (stateless) declarative format that IDEs can load/edit/save directly.

Specific responses follow.

--------------------------------------------------------------------------------

On 01/11/2016 12:24 PM, Charles Huet wrote:
> I think these goals aim towards a faster configure, and the ability to
> only partly reconfigure, right?

Yes.

> I know I am largely biased by the project I work on, but I do not see how
> parallel evaluation woud be a huge benefit.
[snip]
> And how would that work with CMakeLists that affect their parent scope ?

Evaluation of the imperative language is currently serial for reasons like
this, which is why I said it would take semantic changes to enable parallel
evaluation. This is not the main point of my proposal so I'd rather not
get bogged down in the details of this part of the discussion.

>> Ideally most of the specification (sources, libraries, executables, etc.)
>> should be in a pure format that can be evaluated without side effects (e.g.
>> declarative or functional).
>
> I'm not sure I understand how this could be done without losing a lot of
> what CMake offers, such as copying or generating files.

I'm not proposing dropping the current imperative capabilities.

> I'm leaning towards a declarative approach as it is quite easy to learn
> (since you declare objects, and every C++ programmer I know is familiar
> with those)

Yes.

> It seems you are leaning towards pure functional, but I do not see how
> this would work with the current way CMake handles variables and scope,
> could you elaborate ?

While declarative may get us most of the way, advanced users may wish to
hook in to generation-time evaluation. A clean way to do that would be
to specify a function within the declared values. It would not have to
be in a functional language as long as it has no access to anything other
than the inputs passed to it during evaluation.

I mentioned "functional" mostly as an example of a specification whose
evaluation is free of side effects.

> To clarify, only the following lines should be considered when looking at the POC.
>>     myProject=cmake.Project("MyTestProject")
>>     myProject.targets=[cmake.SharedLibrary("testLibrary",["lib.cxx"])]

Yes, this is the kind of stuff that can be in a declarative format.

> It seems you have in mind to write a new CMake language.

No, at most a new specification format that can be used for IDE integration.
If some kind of user-coded function were included in the specification it
should certainly be in an existing language.

> Maybe I should take my POC further

I think implementation even of a POC is premature at this point.  We
should explore the design space further.

> CMake's own buildsystem seems like a good testing ground for this, but
> it is a little too big for a first go, do you know of a small CMake-based
> project that would be better suited ?

Maybe you could find something in our test suite.

> I don't have a clear view of what a pure functional CMake would look like,
> but if you give me some mock code, I could give a try at bringing some pure
> functional language up to the level of my POC and we could use it as a more
> concrete discussion support.

I have no prototype (nor substantial time to spend on design myself) but
I've imagined a declarative format in a well-known syntax (e.g. JSON or one
of the lightweight human-friendly choices).  If generate-time functionality
is needed then code snippets in e.g. Lua could be included.

--------------------------------------------------------------------------------

Petr Kmoch wrote:
> I'd like to voice my opinion as a somewhat advanced CMake user here.

Thanks for joining the discussion with this point of view.

> For me, one of the strongest points of CMake is the fact that its project
> specification is procedural rather than declarative.

Good. We will not be dropping imperative capabilities.

> end result of our framework is that the CMakeLists consist mostly of
> declarative commands from our framework

Yes, many projects have such frameworks. Most of them result in a
declarative spec inside calls to their macros/functions. I'd like to
formalize such specs in a re-usable way.

> If I understand Brad's suggestion correctly, it would amount to a
> (possibly empty) procedural step being used to generate a declarative
> description of the buildsystem.

Not quite. Yes, the procedural/imperative part would still be the entry
point as it is now. However, the declarative part would also be an
input, not an output. The procedural part's role would be to compute
*parameters* to be used for the evaluation of the declarative spec.

For example, imagine the declarative spec somehow encodes that source
file "foo.c" is optional based on some condition "FOO". The value of
that condition could be computed by the procedural part of the process
based on system introspection, command-line options, etc., and then
provided to CMake for use in the final evaluation.

--------------------------------------------------------------------------------

On 01/11/2016 01:21 PM, Pau Garcia i Quiles wrote:
> The moment you make CMake scriptable in more than one language, you are forcing
> every CMake user to learn that additional language because sooner or later he
> will step into a third-party that is using that additional language.

I don't think the main "entry point" language should be selectable.

As discussed above if some kind of callback or user-coded function needs to
be included for advanced usage of the declarative spec then we would need
a language for it.  The current CMake language is not well suited to that
use case (e.g. no expressions or return values), so an existing alternative
language should be chosen.

CMake's current "generator expressions" fill this role somewhat now and are
essentially a sub-language.  As with the main language they grew out of
something not intended to serve their current full role.  They could be
superseded by a common alternative generate-time language too.

> Also, declarative? Why?

See above.

> There are already a few declarative build systems

Yes, and perhaps we can learn from their formats for the proposed declarative
part.

> qbs, one of the reasons for its existence was CMake was not declarative

IIUC that is exactly because of the fact that an imperative spec cannot
be pierced easily for editing in IDEs.

--------------------------------------------------------------------------------

Thanks all for the discussion so far!

-Brad