[CMake] CMake bug tracker discussion

[David Cole]

On Thu, Dec 9, 2010 at 7:09 PM, Alan W. Irwin <irwin at beluga.phys.uvic.ca> wrote:
> On 2010-12-09 17:06-0500 David Cole wrote:
>
>> Hello CMake users and devs,
>>
>> (And now for something completely different...)
>>
>> Controversial questions:
>>
>> - Should we eliminate the bug tracker entirely and just do all
>> discussion and patches on the mailing list? (Why have two sources of
>> information...?)
>>
>> - Or, alternatively, should we eliminate the bulk of mailing list
>> traffic, and insist on issues in the bug tracker being the main
>> conversational forum for the whole community?
>>
>> I'd like to have this discussion here publicly, to try to get a good
>> sense of varous community members attitudes and feelings.
>>
>>
>> I'll start the ball rolling by saying that, personally, I like the bug
>> tracker. I find it much easier to keep a list of issues organized and
>> accessible than I can with email filters and folders. But I still see
>> a need for both tools.
>>
>
> I am glad you brought this subject up because there is a real problem
> engulfing CMake's bug tracker as we speak.  It appears from the
> resolved category of CMake bugs on the bug tracker, that ~10 bugs were
> resolved (not necessarily fixed) in November, but that same month saw
> ~50 new bugs reported (to the cmake-devel mailing list).  That ratio
> of 5 new bugs for every one resolved means the CMake bug tracker is
> rapidly being filled with unresolved issues with no hope of ever
> resolving them unless some substantial changes are made.

A couple of points to alleviate (or at least reduce your concerns)
about this "trend."

(1) It's not as bad as you think based on November alone. In the last
365 days, 589 bugs have been opened and 341 have been resolved. Net
increase in open bugs of 248 over the last year. So ratio of
new-bugs:resolved-bugs is actually about 1.7:1.

(2) Having spent much of my own time perusing bugs in the CMake bug
tracker, I can tell you that the 0.7 over the 1 that makes up that
1.7:1 ratio are noise or duplication or just not worth anyone's time.

So the ratio is more even than you think, and I don't think we need a
major adjustment in this respect.

I've gotta digest the rest of what you said. Maybe more reply to come
later..... :-)


Thanks for the input,
David


>
> Here are some ideas to help reduce that insanely unsustainable ratio
> of five down to a sustainable unity.
>
> 1.  Reduce the number of new bug reports.
>
>  a.  My guess is lots of those new bug reports are noise due to
>      misunderstanding of how to use CMake (despite what I consider to
>      be outstanding documentation). So reduce the numbers by strongly
>      suggesting that all potential bugs should be discussed first on
>      the mailing list (with [BUG?] in the subject line to draw
>      attention to it) to make sure they really are bugs before
>      writing up a bug report.
>
>  b.  Avoid using the bug tracker whenever possible, e.g., quick and
>      obvious fixes. I have experienced other organizations which
>      demanded everything go to the bugtracker where posting to the
>      bugtracker became the point rather than actually fixing bugs. In
>      other words, bug trackers have been used by some organizations
>      as an excuse for inaction on bugs!  This is clearly not the case
>      with the CMake developers (I just this morning had one of my
>      discussed bugs fixed without going through the bug tracker), but
>      I feel strongly about that important pitfall of bugtrackers so I
>      brought it up explicitly here. It should be emphasized that bug
>      trackers can play an important role for the more difficult fixes
>      that take a lot of experimentation and thought to get right, but
>      in my view creating a bug report on a bug tracker has an obvious
>      cost for the reporter and for the bug triage team afterward
>      so should be avoided for the simple stuff.
>
> 2. Increase the resources you spend on resolving bugs in the bug
> tracker.
>
>  a.  More community involvement, e.g., encourage a community bugfix
>      team whose principal job was to do the first-level bug triage
>      such as investigating the questions of whether it is a
>      well-posed bug report (e.g., is there enough information in the
>      bug report), is the issue reproducible, and ultimately is it a
>      real bug or not.  And if not, members of that team would have
>      the power to close the bug as "not a bug".
>
>  b.  A modest increase in Kitware resources devoted to bug fixing.
>      This is obviously a judgement call that is up to Kitware, but if
>      I had a say in Kitware's allocation of resources, this is how I
>      would argue for these additional resources. Kitware doesn't get
>      direct money for their CMake development efforts, but CMake is
>      an enormous boost to their reputation which presumably
>      translates to more commercial success through their paid-for
>      products.  The bottom line is ultimately it hurts Kitware's
>      reputation if their CMake bugtracker is filled to the brim with
>      unresolved issues so some increase in Kitware resources as well
>      as encouraging community involvement is warranted to help deal
>      with the current bad situation.
>
> In sum, I agree with the others who have posted on this question that
> you need both mailing-list discussion of all bugs and the bugtracker.
> My additional ideas beyond that general idea are (1) you should
> encourage mailing list discussion to make sure a CMake problem some
> user is having is really due to a bug, and (b) make a conscious effort
> to make quick fixes when those are warranted, and (c) encourage use of
> the bug tracker only for the more difficult issues where there is
> obviously no quick fix.
>
> Alan
> __________________________
> Alan W. Irwin
>
> Astronomical research affiliation with Department of Physics and Astronomy,
> University of Victoria (astrowww.phys.uvic.ca).
>
> Programming affiliations with the FreeEOS equation-of-state implementation
> for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
> package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
> Linux Links project (loll.sf.net); and the Linux Brochure Project
> (lbproject.sf.net).
> __________________________
>
> Linux-powered Science
> __________________________
>