(0028568)
Brad King (manager)
2012-02-13 08:56

The documentation in find_package *is* the definition.

(0028575)
Dave Abrahams (reporter)
2012-02-13 12:42

That's inadequate, IMO. The reader experience is, "wow, package registry? Neat! What is this thing exactly? Where can I find out more?" (Answer: nowehre). It doesn't even say whether this "registry" information persists across runs of CMake. There's also a "system package registry" somewhere, IIUC, which I've seen mentioned in passing in some web search results. These concepts deserve a few words of introduction on its own.

(0028587)
Brad King (manager)
2012-02-14 15:58

I've written a tutorial in the Wiki here:

http://www.cmake.org/Wiki/CMake/Tutorials/Package_Registry [^]

which can serve as documentation until we have a good place to put this in CMake's reference documentation.

(0028588)
Dave Abrahams (reporter)
2012-02-14 16:21

Thanks; that's a very good start

(0028591)
Daniel Pfeifer (reporter)
2012-02-14 17:39
edited on: 2012-02-14 18:02

This is very good indeed!

From the export(PACKAGE) documentation I knew there was some Package Registry. I did not know that there were actually two different Package Registries. I think this is a very cool feature.

On non-Windows platforms, packages are usually installed to a standard location, so there is no need for a System Package Registry.

I have two questions that are not answered in the tutorial:

1. I can use 'export(PACKAGE)' to add entries to the User Package Registry. How can I add entries to the System Package Registry? Do I have to do this manually or do the NSIS based installers made by CPack already create the entries?

2. Is it possible to store multiple versions of the same package in the repositories?

(0028593)
Brad King (manager)
2012-02-15 09:35

Re 0012958:0028591: I updated the wiki page to answer your questions. I also added a note at the top that this feature was added in CMake 2.8.5.

(0028594)
Daniel Pfeifer (reporter)
2012-02-15 10:33

Thanks.

(0029388)
Dave Abrahams (reporter)
2012-05-02 15:59

More unanswered questions:

* In what order does find_package consider the package registry? For example, if I set PackageName_DIR in the Environment and PackageName is already in the registry, which one does it use?

* Is it possible to tell CMake to ignore the registry?

* How do I clear the registry?

(0029389)
Brad King (manager)
2012-05-02 16:28

Re 0012958:0029388: The find_package documentation already specifies the answers to the first two questions (which IMO are deep enough that reference documentation is a suitable place to answer them). The command documentation enumerates a set of steps followed by the search. As of CMake 2.8.8 steps 6 and 8 are the user and system package registries, respectively. Both are after the environment (step 2). Both steps document an option to find_package to skip them (NO_CMAKE_PACKAGE_REGISTRY and NO_CMAKE_SYSTEM_PACKAGE_REGISTRY). I tweaked the intro section of the tutorial to to reference the find_package documentation more specifically.

For the third question, the tutorial already mentions that find_package will remove stale entries if it has sufficient permissions. I added an explicit section with more detail:

http://www.cmake.org/Wiki/CMake/Tutorials/Package_Registry#Ownership [^]

(0029390)
Dave Abrahams (reporter)
2012-05-02 16:45

Sorry about that; I've been looking at my manpage, which apparently is not up-to-date.

(0029523)
Dave Abrahams (reporter)
2012-05-21 11:21

The problem I'm having here, though, is that I really need to suppress the use of the registry from the CMake command-line. I have a scenario where

a. I need to run cmake twice, to deal with circular dependencies
b. On the second pass, it is barfing on BoostCoreConfigVersion.cmake, which has this stanza generated at its end:

# check that the installed version has the same 32/64bit-ness as the one which is currently searching:
if(NOT "${CMAKE_SIZEOF_VOID_P}" STREQUAL "")
  math(EXPR installedBits " * 8")
  set(PACKAGE_VERSION "${PACKAGE_VERSION} (${installedBits}bit)")
  set(PACKAGE_VERSION_UNSUITABLE TRUE)
endif()

Except the problem is, this is the one pointed to by the registry. The one that's actually part of the project I'm building (via add_subdirectory) ends with:

# check that the installed version has the same 32/64bit-ness as the one which is currently searching:
if(NOT "${CMAKE_SIZEOF_VOID_P}" STREQUAL "8")
  math(EXPR installedBits "8 * 8")
  set(PACKAGE_VERSION "${PACKAGE_VERSION} (${installedBits}bit)")
  set(PACKAGE_VERSION_UNSUITABLE TRUE)
endif()

and that one of course works.

(0029526)
Brad King (manager)
2012-05-21 11:53

Re 0012958:0029523: The package version file should not have a command error in it. I've reported issue 0013241 for that.

Ideally find_package would be able to tolerate such errors and simply assume the version is not suitable. Unfortunately it will take some non-trivial work in CMake internals to achieve that.

(0029527)
Brad King (manager)
2012-05-21 11:56

Re 0012958:0029523: The same problem can occur with any location that find_package searches, not just locations found in the registry. Issue 0013241 created a broken setup. Rather than working around the broken setup with case-specific command-line parameters the setup should be fixed.

(0029528)
Dave Abrahams (reporter)
2012-05-21 11:58

But I have a replacement for the broken piece *right here in my source and build trees*. Shouldn't CMake give priority to that bit over what it finds in the registry?

(0029540)
Brad King (manager)
2012-05-21 19:14

Re 0012958:0029528: Having been bitten by issue 0013241 you can remove the registry entry by hand. If it is put there by the first of your two runs then you can fix that with a workaround that ensures CMAKE_SIZEOF_VOID_P is set.

If you want to tell find_package where to look then set <package>_DIR yourself on the command line. Then it won't search at all.

The find_package command documents many steps it follows in order to do the search. The package registries are not the first on the list. You can set one of the other values like CMAKE_PREFIX_PATH to the location.

(0029541)
Dave Abrahams (reporter)
2012-05-21 19:21

I threw out the whole registry. It's just an optimization, right?

What's the workaround?

I guess the reason find_package was invoked was that my add_subdirectory calls weren't topologically sorted.

Hmm. So one workaround is to set CMAKE_PREFIX_PATH to include all the subdirectories in my source (or is it build?) tree before doing all the add_subdirectory calls? That seems awfully boilerplate-ish, doesn't it?

(0029542)
Brad King (manager)
2012-05-22 08:03

Re 0012958:0029541: The package registry is intended to help a project find its dependencies in arbitrary locations on your disk when they are built/installed in steps that are not known to each other (and probably invoked by hand separately). It is not an optimization. If I build package X in c:/random/directory/X then how is CMake or package Y supposed to know that I put X there? It's not a standard search path. To solve this problem X can add an entry to the package registry. Then CMake has a way to find it.

If you have some common super-project that is managing the build of multiple packages then it can just tell each project where to find its dependencies, because the super-project built them too and knows where it put them. The package registry is not a reliable way to ensure that Y finds the X that the super-project intends it to find because the order among entries for X is undefined.

(0029547)
Dave Abrahams (reporter)
2012-05-22 16:47

Thanks, but I feel like the last three question marks went unanswered here.

(0041986)
Kitware Robot (administrator)
2016-06-10 14:28

Resolving issue as `moved`.

This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

Notes
(0028568) Brad King (manager) 2012-02-13 08:56	The documentation in find_package is the definition.

(0028575) Dave Abrahams (reporter) 2012-02-13 12:42	That's inadequate, IMO. The reader experience is, "wow, package registry? Neat! What is this thing exactly? Where can I find out more?" (Answer: nowehre). It doesn't even say whether this "registry" information persists across runs of CMake. There's also a "system package registry" somewhere, IIUC, which I've seen mentioned in passing in some web search results. These concepts deserve a few words of introduction on its own.

(0028587) Brad King (manager) 2012-02-14 15:58	I've written a tutorial in the Wiki here: http://www.cmake.org/Wiki/CMake/Tutorials/Package_Registry [^] which can serve as documentation until we have a good place to put this in CMake's reference documentation.

(0028588) Dave Abrahams (reporter) 2012-02-14 16:21	Thanks; that's a very good start

(0028591) Daniel Pfeifer (reporter) 2012-02-14 17:39 edited on: 2012-02-14 18:02	This is very good indeed! From the export(PACKAGE) documentation I knew there was some Package Registry. I did not know that there were actually two different Package Registries. I think this is a very cool feature. On non-Windows platforms, packages are usually installed to a standard location, so there is no need for a System Package Registry. I have two questions that are not answered in the tutorial: 1. I can use 'export(PACKAGE)' to add entries to the User Package Registry. How can I add entries to the System Package Registry? Do I have to do this manually or do the NSIS based installers made by CPack already create the entries? 2. Is it possible to store multiple versions of the same package in the repositories?

(0028593) Brad King (manager) 2012-02-15 09:35	Re 0012958:0028591: I updated the wiki page to answer your questions. I also added a note at the top that this feature was added in CMake 2.8.5.

(0028594) Daniel Pfeifer (reporter) 2012-02-15 10:33	Thanks.

(0029388) Dave Abrahams (reporter) 2012-05-02 15:59	More unanswered questions: * In what order does find_package consider the package registry? For example, if I set PackageName_DIR in the Environment and PackageName is already in the registry, which one does it use? * Is it possible to tell CMake to ignore the registry? * How do I clear the registry?

(0029389) Brad King (manager) 2012-05-02 16:28	Re 0012958:0029388: The find_package documentation already specifies the answers to the first two questions (which IMO are deep enough that reference documentation is a suitable place to answer them). The command documentation enumerates a set of steps followed by the search. As of CMake 2.8.8 steps 6 and 8 are the user and system package registries, respectively. Both are after the environment (step 2). Both steps document an option to find_package to skip them (NO_CMAKE_PACKAGE_REGISTRY and NO_CMAKE_SYSTEM_PACKAGE_REGISTRY). I tweaked the intro section of the tutorial to to reference the find_package documentation more specifically. For the third question, the tutorial already mentions that find_package will remove stale entries if it has sufficient permissions. I added an explicit section with more detail: http://www.cmake.org/Wiki/CMake/Tutorials/Package_Registry#Ownership [^]

(0029390) Dave Abrahams (reporter) 2012-05-02 16:45	Sorry about that; I've been looking at my manpage, which apparently is not up-to-date.

(0029523) Dave Abrahams (reporter) 2012-05-21 11:21	The problem I'm having here, though, is that I really need to suppress the use of the registry from the CMake command-line. I have a scenario where a. I need to run cmake twice, to deal with circular dependencies b. On the second pass, it is barfing on BoostCoreConfigVersion.cmake, which has this stanza generated at its end: # check that the installed version has the same 32/64bit-ness as the one which is currently searching: if(NOT "${CMAKE_SIZEOF_VOID_P}" STREQUAL "") math(EXPR installedBits " * 8") set(PACKAGE_VERSION "${PACKAGE_VERSION} (${installedBits}bit)") set(PACKAGE_VERSION_UNSUITABLE TRUE) endif() Except the problem is, this is the one pointed to by the registry. The one that's actually part of the project I'm building (via add_subdirectory) ends with: # check that the installed version has the same 32/64bit-ness as the one which is currently searching: if(NOT "${CMAKE_SIZEOF_VOID_P}" STREQUAL "8") math(EXPR installedBits "8 * 8") set(PACKAGE_VERSION "${PACKAGE_VERSION} (${installedBits}bit)") set(PACKAGE_VERSION_UNSUITABLE TRUE) endif() and that one of course works.

(0029526) Brad King (manager) 2012-05-21 11:53	Re 0012958:0029523: The package version file should not have a command error in it. I've reported issue 0013241 for that. Ideally find_package would be able to tolerate such errors and simply assume the version is not suitable. Unfortunately it will take some non-trivial work in CMake internals to achieve that.

(0029527) Brad King (manager) 2012-05-21 11:56	Re 0012958:0029523: The same problem can occur with any location that find_package searches, not just locations found in the registry. Issue 0013241 created a broken setup. Rather than working around the broken setup with case-specific command-line parameters the setup should be fixed.

(0029528) Dave Abrahams (reporter) 2012-05-21 11:58	But I have a replacement for the broken piece right here in my source and build trees. Shouldn't CMake give priority to that bit over what it finds in the registry?

(0029540) Brad King (manager) 2012-05-21 19:14	Re 0012958:0029528: Having been bitten by issue 0013241 you can remove the registry entry by hand. If it is put there by the first of your two runs then you can fix that with a workaround that ensures CMAKE_SIZEOF_VOID_P is set. If you want to tell find_package where to look then set <package>_DIR yourself on the command line. Then it won't search at all. The find_package command documents many steps it follows in order to do the search. The package registries are not the first on the list. You can set one of the other values like CMAKE_PREFIX_PATH to the location.

(0029541) Dave Abrahams (reporter) 2012-05-21 19:21	I threw out the whole registry. It's just an optimization, right? What's the workaround? I guess the reason find_package was invoked was that my add_subdirectory calls weren't topologically sorted. Hmm. So one workaround is to set CMAKE_PREFIX_PATH to include all the subdirectories in my source (or is it build?) tree before doing all the add_subdirectory calls? That seems awfully boilerplate-ish, doesn't it?

(0029542) Brad King (manager) 2012-05-22 08:03	Re 0012958:0029541: The package registry is intended to help a project find its dependencies in arbitrary locations on your disk when they are built/installed in steps that are not known to each other (and probably invoked by hand separately). It is not an optimization. If I build package X in c:/random/directory/X then how is CMake or package Y supposed to know that I put X there? It's not a standard search path. To solve this problem X can add an entry to the package registry. Then CMake has a way to find it. If you have some common super-project that is managing the build of multiple packages then it can just tell each project where to find its dependencies, because the super-project built them too and knows where it put them. The package registry is not a reliable way to ensure that Y finds the X that the super-project intends it to find because the order among entries for X is undefined.

(0029547) Dave Abrahams (reporter) 2012-05-22 16:47	Thanks, but I feel like the last three question marks went unanswered here.

(0041986) Kitware Robot (administrator) 2016-06-10 14:28	Resolving issue as `moved`. This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.

Issue History
Date Modified	Username	Field	Change
2012-02-12 16:34	Dave Abrahams	New Issue
2012-02-13 08:56	Brad King	Note Added: 0028568
2012-02-13 12:42	Dave Abrahams	Note Added: 0028575
2012-02-13 13:57	Brad King	Status	new => backlog
2012-02-14 15:58	Brad King	Note Added: 0028587
2012-02-14 16:21	Dave Abrahams	Note Added: 0028588
2012-02-14 17:39	Daniel Pfeifer	Note Added: 0028591
2012-02-14 18:02	Daniel Pfeifer	Note Edited: 0028591
2012-02-15 09:35	Brad King	Note Added: 0028593
2012-02-15 10:33	Daniel Pfeifer	Note Added: 0028594
2012-05-02 15:59	Dave Abrahams	Note Added: 0029388
2012-05-02 16:28	Brad King	Note Added: 0029389
2012-05-02 16:45	Dave Abrahams	Note Added: 0029390
2012-05-21 11:21	Dave Abrahams	Note Added: 0029523
2012-05-21 11:50	Brad King	Relationship added	related to 0013241
2012-05-21 11:53	Brad King	Note Added: 0029526
2012-05-21 11:56	Brad King	Note Added: 0029527
2012-05-21 11:58	Dave Abrahams	Note Added: 0029528
2012-05-21 19:14	Brad King	Note Added: 0029540
2012-05-21 19:21	Dave Abrahams	Note Added: 0029541
2012-05-22 08:03	Brad King	Note Added: 0029542
2012-05-22 16:47	Dave Abrahams	Note Added: 0029547
2016-06-10 14:28	Kitware Robot	Note Added: 0041986
2016-06-10 14:28	Kitware Robot	Status	backlog => resolved
2016-06-10 14:28	Kitware Robot	Resolution	open => moved
2016-06-10 14:28	Kitware Robot	Assigned To	=> Kitware Robot
2016-06-10 14:31	Kitware Robot	Status	resolved => closed