[CMake] Contribute Find-module to CMake vs Config-file to upstream

[rleigh at codelibre.net]

On 2018-05-21 18:39, Mateusz Loskot wrote:
> I've been recently trying to update/add Find-modules to CMake:
> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
[…]
> The FindLZ4 discussion basically ended with suggestion from Brad that,
> instead of adding Find-module to CMake, I should approach LZ4 project
> and add Config-file to the library itself.
> Then I argued taht Config-files are more complex and I don't feel like
> asking projects, which I don't maintain myself, to accept the extra 
> complexity.
> Finally, Brad replied that CMake doesn't need LZ4 itself and yet for 
> some
> reason is expected to know all about it: versions, headers, libraries, 
> etc.
> and that it's not scalable to teach CMake about every project in the 
> world.

It really doesn't scale.  I've contributed and provided ongoing fixes 
for a few Find modules out of practical necessity, for projects which 
are effectively incapable of adding a Config-file, either because the 
project was stagnant with no new releases (Xerces, Xalan), or where the 
project was so complex that it was effectively impossible to add 
(Boost).  However, that work should have been done by the upstream.  The 
ongoing maintenance cost has been moved from the upstream development, 
where it belongs, to a volunteer (me).  It's work which I do because it 
provides value to me and my employer so I can use CMake with the 
projects I work on, but is otherwise unjustifiable.  You simply can't 
expect every Find module to keep pace with upstream development without 
a small army of volunteers to keep them all up to date.  If they are 
provided with the upstream releases, they are de facto always up to date 
because they are directly matched to the library.

Note that for several of these projects (e.g. Xerces, libtiff), I have 
actually provided upstream CMake build systems which install 
Config-files, so in the future the Find modules can be deprecated and 
users can move to the Config-file as these releases make their way into 
distributions.  That's semi-altruistic.  I've spent a lot of effort up 
front to make CMake infrastructure in these projects maintainable for 
the long term, and something which can be delegated to others for 
ongoing maintenance.

> This opinion does not surprise me and I understand the rationale.
> I argue that CMake does need to know about every project in the world,
> or every project that CMake users wish to use, directly or indirectly.
> Without the knowledge CMake would simply be useless.

Not useless, that's going a bit far.  It's requiring a little bit of 
extra work to be functional.  For libraries without Find modules, you 
can usually write your own in a couple of dozen lines.  At that point 
your options are:

- to upstream it as a Config-file template to the upstream developers
- to maintain it independently in your project
- to upstream it to cmake.git

Depending upon the library in question, I've done all three at various 
times.

> As a CMake user, I expect to be able to install CMake and get my 
> dependencies
> recognised by CMake as CMake's responsibility

> If a library is strictly GNU Autoconf, I'm on lost position trying to 
> convince
> maitnainers to accept CMake stuff and even if they do, they won't be 
> willing to
> maintain it - broken Config-files may be deployed with new packages.
> That puts users of CMake and such library on lost position:
> CMake says no to Find-module, the library does not care either.

If it's using Autoconf, then it can still provide and install CMake 
configuration files.  It can use @var@ substitutions in the 
Config-file.in template and then install it.  Just like for pkg-config 
templates.

Individual projects may have their own policies, and some might not want 
to have CMake support.  In general though, if you were to provide a 
tested patch that creates the Config-file in AC_OUTPUT and then install 
as e.g. cmake_pkgconfig_DATA in Makefile.am, add a distcheck hook and/or 
CI logic to test it, and most projects are in my experience willing to 
accept the patch.  The only way to find out is to create and submit the 
patch, and have a conversation with them about the cost/benefit of 
including it.  Some might be worried about the maintenance burden, so 
explain the value of it and maybe commit to maintaining it as required.  
Since you'd likely be maintaining it /anyway/, it's not a big task.

> As CMake user I don't want to be left to the discretion of package 
> maintainers.
> Packages may ignore CMake Config-file existence.
> Packages of libraries which provide CMake and alternative build 
> configurations
> may not deploy Config-files or Find-modules.
> 
> IMHO, CMake should encourage contributions of new Find-modules.

As with all things, in my opinion this really comes down to these 
questions: "who is going to do the work", and "who is paying for their 
time".  Contribution also has implied ongoing maintenance obligations.  
If Find modules are contributed and then not maintained, they will 
become broken over time, and who will be expected to keep them updated?  
The CMake core maintainers?  The submitter?  Users who need it?  If it's 
upstreamed, it's much less of a liability.  For the stuff I've 
contributed, it's mostly been done on my paid work time because it has 
value for the work projects, enabling easy building and testing on a 
large range of platforms for both our developers and end users.  But 
when I move onto new projects or a new job, I'll find it difficult to 
justify doing this in my spare time, as much as I might like to.  If you 
take a look at the bug list for e.g. FindBoost, there are a lot of open 
issues, but it's so huge and complex that I don't have the resources 
(different systems) to address all of the defects--that requires 
contributions from the people who can actually test these systems I 
can't.  The huge complexity here squarely belongs upstream.  FindBoost 
is one of the more complex modules, and I can fully sympathise with the 
desire to avoid more of these maintenance nightmares.  It's already a 
large burden upon both people like me, and I suspect also the CMake core 
maintainers in creating, testing and reviewing changes for every Boost 
release and for every platform it's expected to run on.  Multiply that 
by the total number of Find modules (approximately 150 at present).

One thing I do think could be improved is the behaviour of find_package 
and MODULE|NO_MODULE|CONFIG.  In order to allow more transparent 
migration from Find modules to Config-files it would be nice if in the 
absence of any of these flags, it would firstly try the Config-file and 
if it was missing, then fall back to a Find module.  This might have 
some compatibility concerns.  However, if the long term goal is to move 
away from Find modules, then a means of transitioning like this would be 
desirable.  Otherwise, doing this manually in the Find module itself 
might be required.


Regards,
Roger