[CMake] Finding Python3

[Michael Hertling]

On 07/22/2010 10:17 AM, Marcel Loose wrote:
> On Thu, 2010-07-22 at 03:09 +0200, Michael Hertling wrote:
>> On 07/21/2010 10:26 AM, Michael Wild wrote:
>>>
>>> On 21. Jul, 2010, at 9:56 , Marcel Loose wrote:
>>>
>>>> On Tue, 2010-07-20 at 09:18 -0700, Alan W. Irwin wrote:
>>>>> On 2010-07-20 17:12+0200 Michael Hertling wrote:
>>>>>
>>>>>> On 07/20/2010 03:26 AM, Alan W. Irwin wrote:
>>>>>>> On 2010-07-20 00:51+0200 Michael Hertling wrote:
>>>>>>>
>>>>>>>> On 07/18/2010 10:14 PM, Alan W. Irwin wrote:
>>>>>>>>> (1) http://public.kitware.com/Bug/view.php?id=10718 is fixed. 
> In
>>>> my
>>>>>>>>> view this bug has been the source of much CMake find trouble
> for
>>>> a long
>>>>>>>>> time, and I hope the CMake developers make it a high priority
> to
>>>> fix
>>>>>>>>> it for CMake-2.8.3.
>>>>>
>>>>>> If I understand correctly, the intention of 10718 is to denote
>>>> possibly
>>>>>> non-equivalent alternatives after NAMES and use the super-path to
>>>> pick
>>>>>> out one of them.
>>>>>
>>>>> Correct.  This issue has come up several times on the PLplot
> developer
>>>>> list in several contexts (not only Python).  Without the fix, it
>>>>> proves impossible to manipulate the super-PATH to allow CMake to
> find
>>>>> anything later in the NAMES list (normally a lower version) if
>>>>> something earlier (normally a higher version) exists anywhere in
> the
>>>>> super-PATH on your system. The fix to 10718 is to swap the inner
> and
>>>>> outer loops in the CMake code so super-PATH order takes precedence
>>>>> over NAMES order.
>>>>>
>>>>> I believe that solution of 10718 will make life easier for Find
> module
>>>>> maintainers and developers.  Which is why I brought it up in
>>>>> connection with this thread.  However, I don't want to
>>>>> over-concentrate on this one matter at the expense of the rest of
> this
>>>>> important topic which is how to improve the Python Find module. 
> So
>>>>> that is probably enough about 10718 for now.
>>>>>
>>>>> Alan
>>>>> __________________________
>>>>> Alan W. Irwin
>>>>>
>>>>> Astronomical research affiliation with Department of Physics and
>>>> Astronomy,
>>>>> University of Victoria (astrowww.phys.uvic.ca).
>>>>>
>>>>> Programming affiliations with the FreeEOS equation-of-state
>>>> implementation
>>>>> for stellar interiors (freeeos.sf.net); PLplot scientific plotting
>>>> software
>>>>> package (plplot.org); the libLASi project (unifont.org/lasi); the
>>>> Loads of
>>>>> Linux Links project (loll.sf.net); and the Linux Brochure Project
>>>>> (lbproject.sf.net).
>>>>> __________________________
>>>>>
>>>>> Linux-powered Science
>>>>> __________________________
>>>>
>>>> I fully agree with Alan. I brought this up on the mailing list as
> well a
>>>> couple of months ago -- see
>>>> http://www.mail-archive.com/cmake@cmake.org/msg27838.html -- but
> didn't
>>>> open a bug for it. From that mail it should be clear that it's not
> only
>>>> FindPython suffering from this problem, but FindBoost as well. 
>>>>
>>>> Re-reading that thread I saw all kinds of suggested solutions to
> the
>>>> "problem" that sounded much more elaborate and difficult to
> implement
>>>> than the solution you and some other have already suggested: turn
> the
>>>> loop inside-out.
>>>>
>>>> If people a Kitware fear this might cause problems with some
> people's
>>>> builds I would suggest to add a policy for this, which defaults to
> the
>>>> proposed IMHO preferred behaviour: put the paths in the outer loop
> and
>>>> the names in the inner loop.
>>>>
>>>> Just my 2cts,
>>>> Marcel Loose.
>>>
>>> +1 from me.
>>>
>>> Michael
>>
>> Hi Marcel,
>> Hi Michael,
>>
>> in my latest reply to Alan,
>>
>> <http://www.mail-archive.com/cmake@cmake.org/msg30112.html>,
>>
>> I presented a consideration that either searching names-before-paths
> or
>> paths-before-names doesn't matter because all NAMES are equivalent, or
>> there can be be situations which let a paths-before-names search fail,
>> and these situations are not pathological. E.g., it's perfectly legal
>> - for development or testing - to have several python installations in
>> /opt/python, i.e. one has, say, /opt/python/bin/python{2.4,2.5,2.6}.
>> Now, FIND_PROGRAM(... NAMES python python2.6 python2.5 python2.4 ...)
>> will never find python2.5 even if the search goes paths-before-names.
>> The reason is that the alternatives reside in the same directory which
>> is possible right due to their different names, so the super-path, as
>> it is named in 10718, is no suitable mean to pick out one of them.
>>
>> Reading <http://www.mail-archive.com/cmake@cmake.org/msg28912.html>
> and
>> regarding the concerned SystemTools::FindProgram() methods implemented
>> in Source/kwsys/SystemTools.cxx, I'm in doubt if swapping the loops in
>> question is really less elaborate and difficult than fixing the find
>> modules to use the NAMES option in a correct manner. E.g., what's
>> wrong with, roughly,
>>
>> IF(Python_FIND_VERSION_COUNT EQUAL 0)
>>     SET(v "")
>> ELSEIF(Python_FIND_VERSION_COUNT GREATER 1)
>>     SET(v "${Python_FIND_VERSION_MAJOR}.${Python_FIND_VERSION_MINOR}")
>> ELSE()
>>     # Look for a suitable python installation, e.g. supported by
>>     # PYTHON_ROOT, cf. BOOST_ROOT, and extract the minor version.
>> ENDIF()
>> FIND_PROGRAM(PYTHON_EXECUTABLE "python${v}" ...)
>>
>> in a FindPython.cmake? The key is to predict the interpreter's name
>> from the version passed in by FIND_PACKAGE(). Of course, invocations
>> like FIND_PACKAGE(Python 2) are difficult since the minor version is
>> needed, so the module must probably look for a suitable installation
>> by itself, but a PYTHON_ROOT variable might help as BOOST_ROOT does
>> with FindBoost.cmake. Moreover, this approach would allow to request
>> a specific version reliably which is not guaranteed by FIND_PROGRAM()
>> with a hardcoded version list after the NAMES option - with or without
>> a solution to 10718.
>>
>> In summary, my point is: Even if the loops are swapped, we wouldn't
> get
>> a solution that works well in real-world scenarios, so I doubt if it's
>> worth the effort, and the effort won't be trivial. Instead, one should
>> prefer to improve the find modules and get rid of those non-equivalent
>> alternatives after the NAMES option, in particular hardcoded version
>> lists, and freshly developed modules should use FIND_PACKAGES()'s
>> version interface right from the beginning.
>>
>> Regards,
>>
>> Michael
>>
>> P.S.: Adding wildcard support to the find commands would probably ease
>>       the situation significantly as, e.g., FIND_PACKAGE(Python 2 ...)
>>       would lead to FIND_PROGRAM(PYTHON_EXECUTABLE python2.* ...), so
>>       there's no need to figure out the minor version in a tricky way.
>> _______________________________________________
> 
> Hi Michael and others,
> 
> I mostly agree with what your saying. However, IMHO, you refer to a
> "perfect world" situation, where all Find modules properly use VERSION
> to specify a version number and do not abuse NAMES for that.
> 
> I know that the current discussion focuses on FindPython; hence the
> subject ;-). However, in the "real world" quite a number of other Find
> scripts are shipped as part of the CMake distribution that don't follow
> this "perfect" scheme either.
> 
> So the real question should be, I guess: Should CMake be fixed by
> swapping the paths and names loops in the FindXXX() functions (issue
> 10718)? Or should all abusing Find scripts be fixed?

IMO, the very first action to take should be to prevent shortcomings of
current modules from being incorporated in newly developed ones like the
hopefully evolving FindPython.cmake, and apart from that, I would indeed
prefer improvements of the concerned modules to an attacking of 10718 as
the latter won't result in a robust solution and bears considerable risks.

Regards,

Michael