[cmake-developers] Encouraging testing activity of the CModule community by establishing a CModule project

[Alan W. Irwin]

Bill Hoffman and I have been having some e-mail discussions about the best
approach for fixing some of the known CMake module problems, and he has
encouraged me to present my ideas to this list to get some discussion going
on this issue.

To briefly introduce myself, I "discovered" CMake last July and I have been
active since in successfully converting several software projects (PLplot,
libLASi, and FreeEOS) from an autotools-based build system to a CMake-based
build system.  As a result of this fairly intense CMake experience from the
build-system perspective, I have some general concerns about the CMake
modules that are released, and I want to propose a fundamental solution for
those concerns.

Current CMake module problems:

(1) Too many bugs and inconsistencies in the current set of released
modules. I could give you some chapter and verse from my own favorite
examples (they are already in the bug tracker, and I have also reminded Bill
of these cases), and I believe Bill is aware of many other examples.

(2) Many solutions to these module problems are posted to the CMake mailing
list, put into the CMake bug tracker (where they are often lost amongst all
the other bug reports there), or even committed to the CMake CVS repository,
but Bill doesn't know whether a modified module from any of these extra
sources is suitable for release or not so he is naturally reluctant to make
any of these modified modules part of an official release.

(3) When people develop modules to be eventually donated back to CMake, they
use the released modules as models which propagates the bad practices that
are in those modules (see issue (1)).

(4) Because of issue (1), those like me who implement build systems for
software packages using CMake are bypassing the problem by going to these
same extra sources mentioned in issue (2), debugging the issues (if any) in
those "extra" modules ourselves and sending in module bug fixes to the list
or the bug tracker. However, very little of this module development activity
benefits the released modules because of issue (2) which just continues to
propagate issue (1).

Solution:

I call the number of build-system developers who are working on CMake
modules the CModule community, and Bill and I agree the solution to the
above problems is to encourage the spirit, energy, and effectiveness of that
community.

To me the obvious way to do this is to create a CModule project with
independent facilities (especially bug tracker and repository). This project
still must have releases that are coordinated as part of the overall CMake
release so the proposed CModule project would not be independent of CMake.
However, I advocate a separately named project because name recognition is
such an important motivator for community volunteers.

I advocate at least an independent bug tracker and repository for the
proposed CModule project that the core team (see below) for the project can
call their own.  Also, if the CModule repository and bug tracker were well
advertised (again, that name recognition factor), then getting a complete
set of the latest versions of every module is just one repository checkout
away and searching for equivalent bugs is made easier with an independent
bug tracker devoted just to modules. Both of these convenience factors
encourage additional module testing and module bug reporting by CMake users
to improve the reliability of the latest repository versions of the modules
and ultimately the released module versions (issue (1) above).  However,
although such practical factors are obviously important in increasing the
effectiveness of the CModule community, in my opinion the most important
factor affecting the quality of the released CMake modules is the spirit and
energy of that community (see below).

Bill's original idea for encouraging the spirit, energy, and effectiveness
of the CModule community was to recognize volunteer maintainers for each
module with certain reasonable criteria for who got recognized this way.  (I
suggest he posts those criteria separately.) I think Bill's idea is an
excellent step in the right direction, but I thought it was also important
to go one step further with the idea that the individual module maintainers
would be promoted to a core team for the CModule project with full developer
privileges (especially repository commit privilege) if/when their sustained
contributions started transcending temporary individual module issues.

An experienced core team of module developers with full developer privileges
would be of substantial practical use. Each different module maintainer
would require some support in terms of committing their changes as well as
giving advice for difficult problems, etc.  This support of module
maintainers is non-trivial (see my PLplot experience with similar support I
describe below) so core team members could divide up that work.  Also, there
are some module problems that transcend individual modules.  To give some
examples, the core team might want to write a macro to properly combine the
XXX_FIND_QUIETLY flag and XXX_FIND_REQUIRED (and possibly
XXX_FIND_REQUIRED_YYY) flags for FindXXX.cmake modules.  (Inconsistencies in
the way this combination of logic is done is a current sore point with
modules.) The core team might also want to implement a series of ctest tests
to help establish that each released module works correctly on all platforms
to deal with issues (1) and (2) above.  However, modifying each module so it
calls the standard macro at the end to combine XXX_FIND_QUIETLY, etc., flag
logic in a consistent way and implementing ctest tests for each module is a
lot of work that would probably only be practical if there were a
substantial number of core team members to help share that work.  Finally,
you will have some extremely active module developers who work on a number
of different modules in a sustained way. Promoting these active module
developers members to the core team just makes sense instead of slowing down
their activities (and reducing their enthusiasm) by making them wait for
some member of the core team to do each of their commits.

The prospect of promotion to the core team with full developer privileges is
also, in my experience, a large motivational factor for volunteers and has a
profound practical impact on developer commitment to the project. The PLplot
developers normally get started by proposing patches.  So like the
individual module maintainers discussed above, they do not have repository
commit privileges.  I am the member of the PLplot core team that usually
deals with external contributors of patches, and I put a lot of energy into
this activity because I know these contributors are potential core team
members.  So I give them quick response, discuss potential problems with
their patches, etc.  If the external contributor shows sustained interest
and the core team likes their work, we eventually invite them to be members
of that core team with full developer privileges (especially Subversion
commit access).  I have noticed time and again that such promotion
dramatically increases the developer's commitment to PLplot as measured by a
large and sustained increase in their PLplot productivity. I think this
principle (the more you trust developers with full developer privileges, the
more energetically they do the right thing) is the fundamental reason that
PLplot has such a strong all-volunteer community of developers, and I think
this lesson about the motivations of volunteer developers is an important
one for encouraging the spirit, energy, and effectiveness of the CModule
community as well.

Summary:

The above is my vision (based especially on my experience with the
all-volunteer PLplot project) of the best way to encourage the spirit,
energy, and effectiveness of the already existing CModule community to solve
the known module problems.  I support Bill's idea of recognizing individual
maintainers of modules without giving them developer privileges.  However,
to further encourage the testing of modules and to encourage a strong and
active CModule community, I would supplement that idea by establishing a
CModule project with separate facilities (at least a bug tracker and
repository) with promotion of all module maintainers who do good work and
have a sustained interest that transcends more than one module to a CModule
core team with full developer privileges (especially repository commit). My
reading of what Bill has told me is he is _almost_ convinced by the case I
have made, but he is still reserving judgement on going ahead and
establishing a CModule project along the above lines and wants to have a
full discussion here before the final decision is made.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________