[CMake] Finding Python3

[Michael Hertling]

On 07/21/2010 10:26 AM, Michael Wild wrote:
> 
> On 21. Jul, 2010, at 9:56 , Marcel Loose wrote:
> 
>> On Tue, 2010-07-20 at 09:18 -0700, Alan W. Irwin wrote:
>>> On 2010-07-20 17:12+0200 Michael Hertling wrote:
>>>
>>>> On 07/20/2010 03:26 AM, Alan W. Irwin wrote:
>>>>> On 2010-07-20 00:51+0200 Michael Hertling wrote:
>>>>>
>>>>>> On 07/18/2010 10:14 PM, Alan W. Irwin wrote:
>>>>>>> (1) http://public.kitware.com/Bug/view.php?id=10718 is fixed.  In
>> my
>>>>>>> view this bug has been the source of much CMake find trouble for
>> a long
>>>>>>> time, and I hope the CMake developers make it a high priority to
>> fix
>>>>>>> it for CMake-2.8.3.
>>>
>>>> If I understand correctly, the intention of 10718 is to denote
>> possibly
>>>> non-equivalent alternatives after NAMES and use the super-path to
>> pick
>>>> out one of them.
>>>
>>> Correct.  This issue has come up several times on the PLplot developer
>>> list in several contexts (not only Python).  Without the fix, it
>>> proves impossible to manipulate the super-PATH to allow CMake to find
>>> anything later in the NAMES list (normally a lower version) if
>>> something earlier (normally a higher version) exists anywhere in the
>>> super-PATH on your system. The fix to 10718 is to swap the inner and
>>> outer loops in the CMake code so super-PATH order takes precedence
>>> over NAMES order.
>>>
>>> I believe that solution of 10718 will make life easier for Find module
>>> maintainers and developers.  Which is why I brought it up in
>>> connection with this thread.  However, I don't want to
>>> over-concentrate on this one matter at the expense of the rest of this
>>> important topic which is how to improve the Python Find module.  So
>>> that is probably enough about 10718 for now.
>>>
>>> Alan
>>> __________________________
>>> Alan W. Irwin
>>>
>>> Astronomical research affiliation with Department of Physics and
>> Astronomy,
>>> University of Victoria (astrowww.phys.uvic.ca).
>>>
>>> Programming affiliations with the FreeEOS equation-of-state
>> implementation
>>> for stellar interiors (freeeos.sf.net); PLplot scientific plotting
>> software
>>> package (plplot.org); the libLASi project (unifont.org/lasi); the
>> Loads of
>>> Linux Links project (loll.sf.net); and the Linux Brochure Project
>>> (lbproject.sf.net).
>>> __________________________
>>>
>>> Linux-powered Science
>>> __________________________
>>
>> I fully agree with Alan. I brought this up on the mailing list as well a
>> couple of months ago -- see
>> http://www.mail-archive.com/cmake@cmake.org/msg27838.html -- but didn't
>> open a bug for it. From that mail it should be clear that it's not only
>> FindPython suffering from this problem, but FindBoost as well. 
>>
>> Re-reading that thread I saw all kinds of suggested solutions to the
>> "problem" that sounded much more elaborate and difficult to implement
>> than the solution you and some other have already suggested: turn the
>> loop inside-out.
>>
>> If people a Kitware fear this might cause problems with some people's
>> builds I would suggest to add a policy for this, which defaults to the
>> proposed IMHO preferred behaviour: put the paths in the outer loop and
>> the names in the inner loop.
>>
>> Just my 2cts,
>> Marcel Loose.
> 
> +1 from me.
> 
> Michael

Hi Marcel,
Hi Michael,

in my latest reply to Alan,

<http://www.mail-archive.com/cmake@cmake.org/msg30112.html>,

I presented a consideration that either searching names-before-paths or
paths-before-names doesn't matter because all NAMES are equivalent, or
there can be be situations which let a paths-before-names search fail,
and these situations are not pathological. E.g., it's perfectly legal
- for development or testing - to have several python installations in
/opt/python, i.e. one has, say, /opt/python/bin/python{2.4,2.5,2.6}.
Now, FIND_PROGRAM(... NAMES python python2.6 python2.5 python2.4 ...)
will never find python2.5 even if the search goes paths-before-names.
The reason is that the alternatives reside in the same directory which
is possible right due to their different names, so the super-path, as
it is named in 10718, is no suitable mean to pick out one of them.

Reading <http://www.mail-archive.com/cmake@cmake.org/msg28912.html> and
regarding the concerned SystemTools::FindProgram() methods implemented
in Source/kwsys/SystemTools.cxx, I'm in doubt if swapping the loops in
question is really less elaborate and difficult than fixing the find
modules to use the NAMES option in a correct manner. E.g., what's
wrong with, roughly,

IF(Python_FIND_VERSION_COUNT EQUAL 0)
    SET(v "")
ELSEIF(Python_FIND_VERSION_COUNT GREATER 1)
    SET(v "${Python_FIND_VERSION_MAJOR}.${Python_FIND_VERSION_MINOR}")
ELSE()
    # Look for a suitable python installation, e.g. supported by
    # PYTHON_ROOT, cf. BOOST_ROOT, and extract the minor version.
ENDIF()
FIND_PROGRAM(PYTHON_EXECUTABLE "python${v}" ...)

in a FindPython.cmake? The key is to predict the interpreter's name
from the version passed in by FIND_PACKAGE(). Of course, invocations
like FIND_PACKAGE(Python 2) are difficult since the minor version is
needed, so the module must probably look for a suitable installation
by itself, but a PYTHON_ROOT variable might help as BOOST_ROOT does
with FindBoost.cmake. Moreover, this approach would allow to request
a specific version reliably which is not guaranteed by FIND_PROGRAM()
with a hardcoded version list after the NAMES option - with or without
a solution to 10718.

In summary, my point is: Even if the loops are swapped, we wouldn't get
a solution that works well in real-world scenarios, so I doubt if it's
worth the effort, and the effort won't be trivial. Instead, one should
prefer to improve the find modules and get rid of those non-equivalent
alternatives after the NAMES option, in particular hardcoded version
lists, and freshly developed modules should use FIND_PACKAGES()'s
version interface right from the beginning.

Regards,

Michael

P.S.: Adding wildcard support to the find commands would probably ease
      the situation significantly as, e.g., FIND_PACKAGE(Python 2 ...)
      would lead to FIND_PROGRAM(PYTHON_EXECUTABLE python2.* ...), so
      there's no need to figure out the minor version in a tricky way.