[cmake-developers] [CMake 0014809]: FindPythonLibs issue (at least on OS X). Should it use sysconfig?
Mantis Bug Tracker
mantis at public.kitware.com
Sat Mar 15 22:04:42 EDT 2014
The following issue has been SUBMITTED.
======================================================================
http://www.itk.org/Bug/view.php?id=14809
======================================================================
Reported By: Nikolaus Demmel
Assigned To:
======================================================================
Project: CMake
Issue ID: 14809
Category: CMake
Reproducibility: always
Severity: major
Priority: normal
Status: new
======================================================================
Date Submitted: 2014-03-15 22:04 EDT
Last Modified: 2014-03-15 22:04 EDT
======================================================================
Summary: FindPythonLibs issue (at least on OS X). Should it
use sysconfig?
Description:
This is an issue about the way that FindPythonLibs works. In particular on OS X
it seems it is quite broken, as I will detail below. I believe this affects not
just OS X 10.9, where I ran my tests, but OS X in general. Other systems like
Linux seem to also have problems with this module, though I have not run any
tests there. I testet this with cmake release 2.8.12.1, but I think the module
has not changed significantly since.
There is already quite some discussion downstream at homebrew
https://github.com/Homebrew/homebrew/issues/25118, but I will try to summarize
everything here:
It seems that currently FindPythonLibs will only find the correct library and
headers for the 2.7 version of python that is supplied with OS X. Basically, it
always returns `/System/Library/Frameworks/Python.framework/Headers` as headers,
and one `/usr/lib/libpython2.7.dylib`, `/usr/lib/libpython2.6.dylib`, or
`/usr/lib/libpython2.5.dylib`, depending on which version is searched for. These
are the three version shipped with OS X. The homebrew installed versions 2.7 and
3.3 in `/usr/local` are never correctly found. Moreover there is some severe
inconsitency. E.g. PYTHONLIBS_VERSION_STRING always returns 2.7.5, no matter
what version was found / asked.
FindPythonInterp on the other hand seems always correctly identify the first
suitable python executable according to PATH.
In my world, FindPythonLibs should try to find a python executable and ask it
for the correct header and lib location (using distutil.sysconfig). However this
could be quite a significant change to the current behaviour (in particular PATH
would then influence FindPythonLibs), but I think this makes for much more
reasonable results overall. If no (suitable) python executable is found, a
fallback could be implemented. In my testing project there is a proof of concept
implementation of this.
This is in some way related to frameworks. Currently FindPythonLibs uses
CMakeFindFrameworks. One problem with homebrew installed pythons is, that their
frameworks are in `/usr/local/Library/Frameworks`, which is not among the
hardcoded list of framework locations in CMakeFindFrameworks. So a related issue
is that there is no way to globally configure the locations where
CMakeFindFrameworks looks. However, even if there are no extra pythons installed
and we are only considering the ones provided by Apple, FindPythonLibs currently
does not work for any version that is not 2.7. The framework issue is
circumvented if FindPythonLibs askes the executable which headers / libs to use.
Also, this needs a proper fix and saying "Just supply `cmake
-DPYTHON_LIBRARIES=... -DPYTHON_HEADERS=...`" is not a practical solution.
Steps to Reproduce:
Head over to https://github.com/NikolausDemmel/cmake_python_test and follow the
instructions. My output is also linked from there.
Additional Information:
This is just a more or less randomly ordered list of links I came across while
looking into this. Some are more or less related. Some might be fixed long
since, some might not be.
- Homebrew issue: https://github.com/Homebrew/homebrew/issues/25118
- Testing package: https://github.com/NikolausDemmel/cmake_python_test
- 2009 list post, saying that `-framework Python` should not be used:
www.vtk.org/pipermail/vtk-developers/2009-December/006755.html
- 2009 issue related to python and frameworks:
www.cmake.org/pipermail/cmake/2009-November/033136.html
- 2012 problem report, which is likely the same issue as this:
http://cmake.3232098.n2.nabble.com/FindPythonLibs-Version-selection-on-OS-X-td7182880.html
- 2011 stackoverflow post about this issue:
stackoverflow.com/questions/7660001/cmake-finds-wrong-python-libs
- 2012 related issue on freebsd:
lists.freebsd.org/pipermail/freebsd-python/2012-May/004084.html
- homebrew issue about this from 2 years ago:
https://github.com/Homebrew/homebrew/issues/10393, suggesting among others that
there are custom fixes flying around like the following:
https://github.com/Homebrew/homebrew/blob/51d054c/Library/Formula/vtk.rb#L31-49,
or
https://github.com/Homebrew/homebrew/blob/51d054c/Library/Formula/opencv.rb#L50-73
As for other related CMake tickets:
- There is some recent activity around FindPythonInterp and FindPythonLibs
consistency, however IMHO this will not solve out issue here:
http://www.itk.org/Bug/view.php?id=13794
- Another issue marked as duplicate of the above, however I think it will also
not be fixed by the above: http://www.itk.org/Bug/view.php?id=10077
- Something about CMAKE_PREFIX_PATH: http://www.itk.org/Bug/view.php?id=12133
- An issue indicating that something similar had already been suggested as far
back as 2006, but never went anywhere: http://www.itk.org/Bug/view.php?id=2257
======================================================================
Issue History
Date Modified Username Field Change
======================================================================
2014-03-15 22:04 Nikolaus DemmelNew Issue
======================================================================
More information about the cmake-developers
mailing list