[cmake-developers] Using a custom DSL compiler in CMake

[Roger Leigh]

On 21/06/2016 18:18, Chris Bieneman wrote:
> Hello cmake-developers,
>
> I’m trying to find a solution to a long running problem in our build. In LLVM we have a domain specific language named TableGen that we use to generate header files used throughout the project. Our current solution to work with this tool in CMake is pretty terrible for a number of reasons.
>
> The biggest problem is that we don’t have good dependency handling around the inputs and outputs from the DSL. Some of this is caused by our CMake targets not being granular (which I know how to fix). The harder part is that our DSL supports C-like includes.
>
> Our DSL compiler can generate .d files, but hooking that up to CMake is a harder problem. My thought was to try and treat TableGen as a language. There are some complications with that because we don’t actually have a compiler for it at configuration time.
>
> Any thoughts and guidance would be greatly appreciated. I feel as if I’m about to fall into a deep and scary rabbit hole.

In one of the projects I maintain at work, we generate a lot of 
different C++ sources and headers using a custom generator tool.  It 
consumes an XML schema, and then processes it to turn that formal data 
model into a set of C++ classes, enums and serialisation functions, 
using a series of templates.

I needed to make this work seamlessly with CMake, and so I wrote a set 
of functions to introspect the dependencies and then generate all the 
necessary targets with appropriate dependencies.  While I can't claim 
this is going to be the best use of CMake in the world, it's quite 
functional and does its job well.

 
https://github.com/openmicroscopy/bioformats/blob/develop/cpp/cmake/XsdFu.cmake

This contains helper macros to do the introspection.  We basically run 
the generator twice for each "target" using --print-generated and 
--print-depends options.  These cause the corresponding files to be 
dumped to stdout.  We then use these file lists as the OUTPUT and 
DEPENDS lists for add_custom_command.  NB. This tool outputs multiple 
source files at once which all go into the OUTPUT.

You can then see those macros in use here:
 
https://github.com/openmicroscopy/bioformats/blob/develop/cpp/lib/ome/xml/CMakeLists.txt#L42
where we then add those outputs to a regular library target.

While I think this is too custom to be of direct use, it might be useful 
for inspiration.  The essential point is to introspect the dependencies 
and outputs as you run cmake (which can emit the includes you mentioned 
as dependencies), and then use that information to generate the 
appropriate custom commands.  Everything will then be built at build 
time, in parallel.  How you do the introspection is entirely up to you. 
I added support to the generator itself since it already had the 
relevant information (it's a python tool) and was quick and easy to add. 
  You could also parse the files directly with cmake, or with a custom 
script in any language you like.


Hope that's at least provided some ideas!


Regards,
Roger