[cmake-developers] CMake alternative language

[Charles Huet]

Hi,

> * There is a lot of code out there in the current CMake language so I do
not
  think it is realistic to drop it.  I'm not proposing that this change.

I am. (more below)

> * Many projects build elaborate macro/function systems in the CMake
language
  in order to end up with a declarative specification listing the actual
  source files, dependencies, and usage requirements.  I'd like to offer
  an alternative to this.

In my experience, most of the elaborate macros/functions come either from a
misunderstanding of some of CMake's internals (scope, link debug/release
libs, etc) or to circumvent some shortcoming of the CMake language (e.g. no
return value for functions).

>  I'd like to improve this by *optionally* moving part of the specification
  to a (stateless) declarative format that IDEs can load/edit/save directly

Split the buildsystem in two different languages ? Would the declarative
part be in a different file ?
Also, the declarative part in my opinion must take advantage of the
language.
For instance, add a source file only for WIN32 systems should be easy in
said declarative format.
Using a custom language (based on JSON for instance) would mean to add
conditionals, which comes back to making a custom language again.


To come back to my first point, I understand completely that this would be
a tremendous change, and the transition would be difficult to say the
least. But I think it would be more than worth it in the long term.

> The moment you make CMake scriptable in more than one language, you are
forcing
> every CMake user to learn that additional language because sooner or
later he
> will step into a third-party that is using that additional language.

What I have in mind is to deprecate the current CMake language and replace
it with another language. So there would be a transition period, but in the
end there would only be one language again, and a better one.

If CMake transitioned to python (or Lua, or whatever) newcomers to CMake
would not need learn a new language (or at least learn one that has many
resources, and can be used for other purposes).
Old-timers would have to learn a new language (or maybe not, most
programmers I know have played a bit with python to automate simple tasks),
but this would be easier than learning CMake was, since there are
established rules and a more consistent design to it.

Of course I'm not saying this should happen overnight, nor am I saying this
*must* happen, but I think discussing it can only be beneficial.

I've seen lots of people wonder how to make their CMake scripts more
efficient, and currently this is a very difficult thing to do, since there
is no profiling possible.
And who can say they never spent way too much time trying to understand why
a variable was not correctly initialized ? When the configure step takes
about 30 seconds, and all you can do is use MESSAGE() to find what happens,
this is no walk in the park. A real debugger would do a world of good to
CMake.
I have seen some hardly understandable CMake code, and only thanks to the
git history was I able to understand that the person who wrote the script
completely misunderstood the CMake language.

>As discussed above if some kind of callback or user-coded function needs to
be included for advanced usage of the declarative spec then we would need
a language for it.  The current CMake language is not well suited to that
use case (e.g. no expressions or return values), so an existing alternative
language should be chosen.

>CMake's current "generator expressions" fill this role somewhat now and are
essentially a sub-language.  As with the main language they grew out of
something not intended to serve their current full role.  They could be
superseded by a common alternative generate-time language too.

These points are part of the reason I think a new language should be used,
if it can cover all of these issues. I'd rather not see a new CMake
declarative language that might itself grow out and become something
difficult to grasp.
Something like generator expressions could be expressed in a language such
as python or lua, by using objects that get resolved at generate time (or
functions, or whatever).


Cheers

Le lun. 11 janv. 2016 à 21:53, Brad King <brad.king at kitware.com> a écrit :

> Hi Folks,
>
> I'm replying directly to my previous post in this thread in order to
> consolidate
> responses to related discussion raised in others' responses to it:
>
>
> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15339/focus=15383
>
> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15339/focus=15386
>
> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15339/focus=15389
>
> General comments:
>
> * There is a lot of code out there in the current CMake language so I do
> not
>   think it is realistic to drop it.  I'm not proposing that this change.
>
> * CMake's procedural/imperative design is good as the main entry point to
>   configuration of a project.  It can do system introspection, file
> generation,
>   etc.  I'm not proposing that this change.
>
> * Many projects build elaborate macro/function systems in the CMake
> language
>   in order to end up with a declarative specification listing the actual
>   source files, dependencies, and usage requirements.  I'd like to offer
>   an alternative to this.
>
> * Integration with IDEs is currently based on one-way generation (VS IDE
>   projects, Xcode projects, CodeBlocks, etc.).  Editing the project build
>   specification requires editing CMake code directly because IDEs cannot
>   easily pierce CMake's procedural/imperative specification:
>
>     https://cmake.org/pipermail/cmake-developers/2016-January/027386.html
>
>   I'd like to improve this by *optionally* moving part of the specification
>   to a (stateless) declarative format that IDEs can load/edit/save
> directly.
>
> Specific responses follow.
>
>
> --------------------------------------------------------------------------------
>
> On 01/11/2016 12:24 PM, Charles Huet wrote:
> > I think these goals aim towards a faster configure, and the ability to
> > only partly reconfigure, right?
>
> Yes.
>
> > I know I am largely biased by the project I work on, but I do not see how
> > parallel evaluation woud be a huge benefit.
> [snip]
> > And how would that work with CMakeLists that affect their parent scope ?
>
> Evaluation of the imperative language is currently serial for reasons like
> this, which is why I said it would take semantic changes to enable parallel
> evaluation. This is not the main point of my proposal so I'd rather not
> get bogged down in the details of this part of the discussion.
>
> >> Ideally most of the specification (sources, libraries, executables,
> etc.)
> >> should be in a pure format that can be evaluated without side effects
> (e.g.
> >> declarative or functional).
> >
> > I'm not sure I understand how this could be done without losing a lot of
> > what CMake offers, such as copying or generating files.
>
> I'm not proposing dropping the current imperative capabilities.
>
> > I'm leaning towards a declarative approach as it is quite easy to learn
> > (since you declare objects, and every C++ programmer I know is familiar
> > with those)
>
> Yes.
>
> > It seems you are leaning towards pure functional, but I do not see how
> > this would work with the current way CMake handles variables and scope,
> > could you elaborate ?
>
> While declarative may get us most of the way, advanced users may wish to
> hook in to generation-time evaluation. A clean way to do that would be
> to specify a function within the declared values. It would not have to
> be in a functional language as long as it has no access to anything other
> than the inputs passed to it during evaluation.
>
> I mentioned "functional" mostly as an example of a specification whose
> evaluation is free of side effects.
>
> > To clarify, only the following lines should be considered when looking
> at the POC.
> >>     myProject=cmake.Project("MyTestProject")
> >>     myProject.targets=[cmake.SharedLibrary("testLibrary",["lib.cxx"])]
>
> Yes, this is the kind of stuff that can be in a declarative format.
>
> > It seems you have in mind to write a new CMake language.
>
> No, at most a new specification format that can be used for IDE
> integration.
> If some kind of user-coded function were included in the specification it
> should certainly be in an existing language.
>
> > Maybe I should take my POC further
>
> I think implementation even of a POC is premature at this point.  We
> should explore the design space further.
>
> > CMake's own buildsystem seems like a good testing ground for this, but
> > it is a little too big for a first go, do you know of a small CMake-based
> > project that would be better suited ?
>
> Maybe you could find something in our test suite.
>
> > I don't have a clear view of what a pure functional CMake would look
> like,
> > but if you give me some mock code, I could give a try at bringing some
> pure
> > functional language up to the level of my POC and we could use it as a
> more
> > concrete discussion support.
>
> I have no prototype (nor substantial time to spend on design myself) but
> I've imagined a declarative format in a well-known syntax (e.g. JSON or one
> of the lightweight human-friendly choices).  If generate-time functionality
> is needed then code snippets in e.g. Lua could be included.
>
>
> --------------------------------------------------------------------------------
>
> Petr Kmoch wrote:
> > I'd like to voice my opinion as a somewhat advanced CMake user here.
>
> Thanks for joining the discussion with this point of view.
>
> > For me, one of the strongest points of CMake is the fact that its project
> > specification is procedural rather than declarative.
>
> Good. We will not be dropping imperative capabilities.
>
> > end result of our framework is that the CMakeLists consist mostly of
> > declarative commands from our framework
>
> Yes, many projects have such frameworks. Most of them result in a
> declarative spec inside calls to their macros/functions. I'd like to
> formalize such specs in a re-usable way.
>
> > If I understand Brad's suggestion correctly, it would amount to a
> > (possibly empty) procedural step being used to generate a declarative
> > description of the buildsystem.
>
> Not quite. Yes, the procedural/imperative part would still be the entry
> point as it is now. However, the declarative part would also be an
> input, not an output. The procedural part's role would be to compute
> *parameters* to be used for the evaluation of the declarative spec.
>
> For example, imagine the declarative spec somehow encodes that source
> file "foo.c" is optional based on some condition "FOO". The value of
> that condition could be computed by the procedural part of the process
> based on system introspection, command-line options, etc., and then
> provided to CMake for use in the final evaluation.
>
>
> --------------------------------------------------------------------------------
>
> On 01/11/2016 01:21 PM, Pau Garcia i Quiles wrote:
> > The moment you make CMake scriptable in more than one language, you are
> forcing
> > every CMake user to learn that additional language because sooner or
> later he
> > will step into a third-party that is using that additional language.
>
> I don't think the main "entry point" language should be selectable.
>
> As discussed above if some kind of callback or user-coded function needs to
> be included for advanced usage of the declarative spec then we would need
> a language for it.  The current CMake language is not well suited to that
> use case (e.g. no expressions or return values), so an existing alternative
> language should be chosen.
>
> CMake's current "generator expressions" fill this role somewhat now and are
> essentially a sub-language.  As with the main language they grew out of
> something not intended to serve their current full role.  They could be
> superseded by a common alternative generate-time language too.
>
> > Also, declarative? Why?
>
> See above.
>
> > There are already a few declarative build systems
>
> Yes, and perhaps we can learn from their formats for the proposed
> declarative
> part.
>
> > qbs, one of the reasons for its existence was CMake was not declarative
>
> IIUC that is exactly because of the fact that an imperative spec cannot
> be pierced easily for editing in IDEs.
>
>
> --------------------------------------------------------------------------------
>
> Thanks all for the discussion so far!
>
> -Brad
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake-developers/attachments/20160113/0f522ee9/attachment-0001.html>