[CMake] Finding Python3

[Michael Wild]

On 22. Jul, 2010, at 3:09 , Michael Hertling wrote:

> On 07/21/2010 10:26 AM, Michael Wild wrote:
>> 
>> On 21. Jul, 2010, at 9:56 , Marcel Loose wrote:
>> 
>>> On Tue, 2010-07-20 at 09:18 -0700, Alan W. Irwin wrote:
>>>> On 2010-07-20 17:12+0200 Michael Hertling wrote:
>>>> 
>>>>> On 07/20/2010 03:26 AM, Alan W. Irwin wrote:
>>>>>> On 2010-07-20 00:51+0200 Michael Hertling wrote:
>>>>>> 
>>>>>>> On 07/18/2010 10:14 PM, Alan W. Irwin wrote:
>>>>>>>> (1) http://public.kitware.com/Bug/view.php?id=10718 is fixed.  In
>>> my
>>>>>>>> view this bug has been the source of much CMake find trouble for
>>> a long
>>>>>>>> time, and I hope the CMake developers make it a high priority to
>>> fix
>>>>>>>> it for CMake-2.8.3.
>>>> 
>>>>> If I understand correctly, the intention of 10718 is to denote
>>> possibly
>>>>> non-equivalent alternatives after NAMES and use the super-path to
>>> pick
>>>>> out one of them.
>>>> 
>>>> Correct.  This issue has come up several times on the PLplot developer
>>>> list in several contexts (not only Python).  Without the fix, it
>>>> proves impossible to manipulate the super-PATH to allow CMake to find
>>>> anything later in the NAMES list (normally a lower version) if
>>>> something earlier (normally a higher version) exists anywhere in the
>>>> super-PATH on your system. The fix to 10718 is to swap the inner and
>>>> outer loops in the CMake code so super-PATH order takes precedence
>>>> over NAMES order.
>>>> 
>>>> I believe that solution of 10718 will make life easier for Find module
>>>> maintainers and developers.  Which is why I brought it up in
>>>> connection with this thread.  However, I don't want to
>>>> over-concentrate on this one matter at the expense of the rest of this
>>>> important topic which is how to improve the Python Find module.  So
>>>> that is probably enough about 10718 for now.
>>>> 
>>>> Alan
>>>> __________________________
>>>> Alan W. Irwin
>>>> 
>>>> Astronomical research affiliation with Department of Physics and
>>> Astronomy,
>>>> University of Victoria (astrowww.phys.uvic.ca).
>>>> 
>>>> Programming affiliations with the FreeEOS equation-of-state
>>> implementation
>>>> for stellar interiors (freeeos.sf.net); PLplot scientific plotting
>>> software
>>>> package (plplot.org); the libLASi project (unifont.org/lasi); the
>>> Loads of
>>>> Linux Links project (loll.sf.net); and the Linux Brochure Project
>>>> (lbproject.sf.net).
>>>> __________________________
>>>> 
>>>> Linux-powered Science
>>>> __________________________
>>> 
>>> I fully agree with Alan. I brought this up on the mailing list as well a
>>> couple of months ago -- see
>>> http://www.mail-archive.com/cmake@cmake.org/msg27838.html -- but didn't
>>> open a bug for it. From that mail it should be clear that it's not only
>>> FindPython suffering from this problem, but FindBoost as well. 
>>> 
>>> Re-reading that thread I saw all kinds of suggested solutions to the
>>> "problem" that sounded much more elaborate and difficult to implement
>>> than the solution you and some other have already suggested: turn the
>>> loop inside-out.
>>> 
>>> If people a Kitware fear this might cause problems with some people's
>>> builds I would suggest to add a policy for this, which defaults to the
>>> proposed IMHO preferred behaviour: put the paths in the outer loop and
>>> the names in the inner loop.
>>> 
>>> Just my 2cts,
>>> Marcel Loose.
>> 
>> +1 from me.
>> 
>> Michael
> 
> Hi Marcel,
> Hi Michael,
> 
> in my latest reply to Alan,
> 
> <http://www.mail-archive.com/cmake@cmake.org/msg30112.html>,
> 
> I presented a consideration that either searching names-before-paths or
> paths-before-names doesn't matter because all NAMES are equivalent, or
> there can be be situations which let a paths-before-names search fail,
> and these situations are not pathological. E.g., it's perfectly legal
> - for development or testing - to have several python installations in
> /opt/python, i.e. one has, say, /opt/python/bin/python{2.4,2.5,2.6}.
> Now, FIND_PROGRAM(... NAMES python python2.6 python2.5 python2.4 ...)
> will never find python2.5 even if the search goes paths-before-names.
> The reason is that the alternatives reside in the same directory which
> is possible right due to their different names, so the super-path, as
> it is named in 10718, is no suitable mean to pick out one of them.
> 
> Reading <http://www.mail-archive.com/cmake@cmake.org/msg28912.html> and
> regarding the concerned SystemTools::FindProgram() methods implemented
> in Source/kwsys/SystemTools.cxx, I'm in doubt if swapping the loops in
> question is really less elaborate and difficult than fixing the find
> modules to use the NAMES option in a correct manner. E.g., what's
> wrong with, roughly,
> 
> IF(Python_FIND_VERSION_COUNT EQUAL 0)
>    SET(v "")
> ELSEIF(Python_FIND_VERSION_COUNT GREATER 1)
>    SET(v "${Python_FIND_VERSION_MAJOR}.${Python_FIND_VERSION_MINOR}")
> ELSE()
>    # Look for a suitable python installation, e.g. supported by
>    # PYTHON_ROOT, cf. BOOST_ROOT, and extract the minor version.
> ENDIF()
> FIND_PROGRAM(PYTHON_EXECUTABLE "python${v}" ...)
> 
> in a FindPython.cmake? The key is to predict the interpreter's name
> from the version passed in by FIND_PACKAGE(). Of course, invocations
> like FIND_PACKAGE(Python 2) are difficult since the minor version is
> needed, so the module must probably look for a suitable installation
> by itself, but a PYTHON_ROOT variable might help as BOOST_ROOT does
> with FindBoost.cmake. Moreover, this approach would allow to request
> a specific version reliably which is not guaranteed by FIND_PROGRAM()
> with a hardcoded version list after the NAMES option - with or without
> a solution to 10718.
> 
> In summary, my point is: Even if the loops are swapped, we wouldn't get
> a solution that works well in real-world scenarios, so I doubt if it's
> worth the effort, and the effort won't be trivial. Instead, one should
> prefer to improve the find modules and get rid of those non-equivalent
> alternatives after the NAMES option, in particular hardcoded version
> lists, and freshly developed modules should use FIND_PACKAGES()'s
> version interface right from the beginning.
> 
> Regards,
> 
> Michael
> 
> P.S.: Adding wildcard support to the find commands would probably ease
>      the situation significantly as, e.g., FIND_PACKAGE(Python 2 ...)
>      would lead to FIND_PROGRAM(PYTHON_EXECUTABLE python2.* ...), so
>      there's no need to figure out the minor version in a tricky way.

I agree with you partially, however: More often than not, I don't care what exact version of Python I get, as long as it's e.g. 2.3 < version < 3.0. But what I would REALLY like, is to get the newest version that is installed and fits in the range. Of course, the user could simply set PYTHON_EXECUTABLE himself...

Which brings me to another problem of find_XXX. Assume you found a way to ensure that your FindPython.cmake finds a consistent set of interpreter, libraries and headers. Now, after running CMake your user doesn't like the interpreter you picked up and changes its path in the cache, however doesn't bother with the libraries and headers. Should we treat this as a user-error, or should such a change cause the libraries and include path to be rediscovered?

Michael