[CMake] providing library information, what's the cmake way

[Johannes Zarl]

On 11/30/2010 at 11:39, Michael Hertling <mhertling at online.de> wrote: 
>> Does this mean that XXX_LIBRARIES etc. should normally incorporate the
>> settings for the components as well? [...]
> 
> Yes, of course, if FIND_PACKAGE(XXX ...) returns successfully one would
> expect XXX_LIBRARIES to hold all libraries necessary to link against in
> order to use all components enabled by the FIND_PACKAGE() invocation,
> i.e. the libraries provided by the components themselves as well as
> their prerequisites.

About that "one would expect": I would be rather stumped and scratching 
my head if I saw this behaviour. But I'll get to that below...

>> [...] IMO this can't work if you call
>> find_package several times with different components. [...]
> 
> This is one of those questions to deal with when it comes to multi-
> component packages: Does FIND_PACKAGE(XXX ...) act accumulatively?

Yes. Again, for a detailed explanation see below...

>> [...] Also, this would 
>> make it impossible to link to the "base library" alone, i.e. without
>> the components...
> 
> One might consider the base library as a component by itself, perhaps
> enabled automatically by resolving inter-component dependencies. As an
> alternative, one could a priori populate XXX_LIBRARIES et al. with the
> base library stuff and add the components' contributions as required.
> Furthermore, a package doesn't need to have a dedicated base library,
> e.g. Boost hasn't, and if you don't specify any components the
> Boost_LIBRARIES variable remains empty.

Which is to be expected, given that Boost is mostly a headers-only 
library. Still, after your find_package call for Boost alone, you can
use all Boost "core" libraries.


-- 
Johannes Zarl
Virtual Reality Services

Johannes Kepler University
Informationsmanagement

Altenbergerstrasze 69
4040 Linz, Austria
Phone: +43 732 2468-8321
johannes.zarl at jku.at
http://vrc.zid.jku.at










>>> - Handling of inter-component dependencies: Imagine the user just says
>>> FIND_PACKAGE(XXX COMPONENTS ZZZ), but ZZZ needs YYY. If YYY is to be
>>> enabled automatically the config file must know about the dependency
>>> and cannot simply add the ZZZ-specific variables to the invocation-
>>> specific ones; the YYY-specific variables must be mentioned, too.
>> 
>> I don't really see this as a problem. Although the code at hand lacks 
>> support for this kind of dependencies, in most cases you won't need it.
>> And if you as the config file writer need it, you obviously know you
>> have dependencies and that you have to write code supporting those
>> dependencies.
> 
> Usually, handling of such dependencies means enabling some components
> the user hasn't requested explicitly, so it's quite straight forward,
> in fact. However, I've mentioned these dependencies to point out that
> the invocation-specific variables like XXX_LIBRARIES sometimes can't
> be assembled by a simple iteration over XXX_FIND_COMPONENTS; instead,
> the dependencies must be resolved first. BTW, do you know an example
> with a component A depending on B only if C is enabled? ;)

No, but you are posing questions that make my head hurt (in a good-ish
way ;-)

>>> - Do multiple consecutive FIND_PACKAGE(XXX ...) invocations act in an
>>> accumulative manner on the invocation-specific variables? 
>> 
>> Generally speaking, I would say: yes. At least this is the way of the
>> least surprise for the user, as in sufficiently complex projects a
>> package may be included at different places with different arguments.
> 
> For the same reason, I'd tend to the opposite; look at the following:
> 
> FIND_PACKAGE(XXX COMPONENTS YYY)
> ...
> ADD_SUBDIRECTORY(subdir)
> 
> In subdir/CMakeLists.txt:
> 
> FIND_PACKAGE(XXX COMPONENTS ZZZ)
> ...
> TARGET_LINK_LIBRARIES(... ${XXX_LIBRARIES})
> 
> Here, the target is also linked against XXX_YYY_LIBRARY as the latter is
> inherited via XXX_LIBRARIES from the parent directory, but if the target
> only needs XXX_ZZZ_LIBRARY it will possibly be overlinked. Although this
> can be avoided by resetting XXX_LIBRARIES before each FIND_PACKAGE()
> invocation, I don't think that's the way one would like to go.
> 
> IMO, the invocation-specific results of any FIND_PACKAGE() call should
> depend solely on the parameters passed in and the well-known variables
> like CMAKE_PREFIX_PATH. The downside is that one must possibly process
> or save the results before they could be overwritten, e.g.:
> 
> FIND_PACKAGE(XXX REQUIRED YYY)
> SET(LIBRARIES ${XXX_LIBRARIES})
> FIND_PACKAGE(XXX COMPONENTS ZZZ)
> LIST(APPEND LIBRARIES ${XXX_LIBRARIES})
> ...
> TARGET_LINK_LIBRARIES(... ${LIBRARIES})
> 
> Since such related calls to FIND_PACKAGE(XXX ...) usually occur nearby
> each other within the same CMakeLists.txt, this little penalty should
> be acceptable whereas accumulated XXX_LIBRARIES and the like may have
> far reaching and surprising effects, especially in complex projects.

Funny enough, my example is almost the same:

FIND_PACKAGE(XXX COMPONENTS YYY)
...
ADD_SUBDIRECTORY(subdir)
...
TARGET_LINK_LIBRARIES(AAA ${XXX_LIBRARIES})
TARGET_LINK_LIBRARIES(BBB ${XXX_LIBRARIES} ${XXX_YYY_LIBRARIES})

In subdir/CMakeLists.txt:

FIND_PACKAGE(XXX COMPONENTS ZZZ)
...
TARGET_LINK_LIBRARIES(subBBB ${XXX_LIBRARIES} ${XXX_ZZZ_LIBRARIES})

As I mentioned above, I would expect XXX_LIBRARIES to contain only the
base library, and find_package calls to act accumulatively.

If it was otherwise, then target AAA would be overlinked, and BBB would 
probably fail to link. Even without the ADD_SUBDIRECTORY the situation
would be far from ideal: in order to avoid overlinking of BBB I have to 
make an additional find_package invocation. In a worst-case scenario one
would need multiple find_package invocations for the same packages before
each target!

The "accumulative scenario" of find_package without side-effects of the
components on the XXX_LIBRARIES is far more transparent:
If you link against XXX_LIBRARIES, you know exactly what you are linking 
against, regardless of who called find_package(XXX) with whatever 
components since you did your find_package call. You don't even have to 
backup your XXX_LIBRARIES and other variables, because they are not altered.

>>> FIND_PACKAGE(XXX REQUIRED YYY)
>>> FIND_PACKAGE(XXX COMPONENTS ZZZ)
>> 
>> Ah, this is how it's done. I was wondering why a call like
>> 
>> find_package(XXX COMPONENTS ZZZ REQUIRED YYY)
>> 
>> sets both XXX_YYY_REQUIRED and XXX_ZZZ_REQUIRED.
> 
> It's XXX_FIND_REQUIRED_{YYY,ZZZ} you talk about, I suppose, and these
> variables are set if the particular component has been mentioned in the
> FIND_PACKAGE() regardless whether the REQUIRED flag is specified or not.

Yes, those are the variables I meant. In the thread you linked to, there
is a post about the history of those variables. If I had read that 
beforehand, I would have known that those variables are just a little
misnamed *g*

>>> When turning towards find modules, the situation becomes even more
>>> complicated. From a user's perspective, find modules and config files
>>> should behave the same, but the formers can't know which components are
>>> available, so they must look for them; this gives rise to questions like:
>>>
>>> - Meaning of the REQUIRED and QUIET options: When a find module looks
>>> for any component the effects of REQUIRED and QUIET depend on whether
>>> the component has been requested explicitly or not. If it hasn't, the
>>> find module mustn't bail out if the component isn't found even though
>>> REQUIRED was specified; that's the opposite of what is expected for an
>>> explicitly requested component. E.g., if FIND_PACKAGE(XXX REQUIRED YYY)
>>> also looks for ZZZ on its own behalf the find module is not expected to
>>> throw a fatal error if ZZZ is absent whereas the absence of YYY should
>>> result in such a behaviour right away. A similar view may be taken for
>>> the QUIET option: Even if it wasn't specified the user wouldn't expect
>>> any messages related to components that weren't requested explicitly.
>> 
>> This isn't that complicated. I have written such a FindLibrary.cmake module
>> (although without dependencies) and the REQUIRED and QUIET options are
>> quite easy to handle. If you split the process into 2 parts, you lose much
>> of the complexity: first, search for any components you can find, second
>> check for the required ones, giving diagnostic/error messages as is
>> appropriate.
> 
> Using two loops would indeed allow to handle the REQUIRED/QUIET flags
> easily, but I would like to apply FIND_PACKAGE_HANDLE_STANDARD_ARGS()
> to components in order to process them in the same standardized and
> unique manner as FPHSA does with single-component packages. This is
> even more interesting as FPHSA has been enhanced recently and is now
> quite complicated so it's not an option anymore to provide a similar
> function for components only. Since FPHSA takes care of setting the
> FOUND variable, issuing messages and bailing out if necessary at one
> go, the two-loop approach doesn't suit, IMO. Instead, I'd prefer to
> promote the REQUIRED/QUIET flags to XXX_YYY_FIND_{REQUIRED,QUIETLY}
> - if YYY has been requested explicitly - and call FPHSA like, e.g.:
> 
> FIND_PACKAGE_HANDLE_STANDARD_ARGS(
>     XXX_YYY
>     DEFAULT_MSG
>     XXX_YYY_LIBRARY XXX_YYY_INCLUDE_DIR
> )
> 
> IF YYY hasn't been requested explicitly, but the find module despite
> looks for it, XXX_YYY_FIND_{REQUIRED,QUIETLY} should be set to FALSE
> and TRUE, resp., so FPHSA handles the REQUIRED/QUIET flags for each
> component as the user probably expects, and XXX_YYY_FOUND is set up
> properly at the same time.

I'm not completely sure if I understood that all correctly. As far as
I can see, the REQUIRED/QUIET flags are set for all components or 
nothing. The interface of find_package simply doesn't allow a more fine
grained setting of those flags.

Regarding FIND_PACKAGE_HANDLE_STANDARD_ARGS: why not simply adding 
another command? In fact, let's discuss its interface right now, and
then implement it:

set(XXX_COMPONENTS "YYY;ZZZ")
FIND_PACKAGE_COMPONENTS_HANDLE_STANDARD_ARGS(
   XXX
   XXX_COMPONENTS
   DEFAULT_MSG
   DEFAULT_SUFFIXES
)

Assuming DEFAULT_SUFFIXES to be "LIBRARIES;INCLUDE_DIRS;DEFINITIONS",
this would check the following variables:
XXX_LIBRARIES
XXX_INCLUDE_DIRS
XXX_DEFINITIONS
XXX_YYY_LIBRARIES
XXX_YYY_INCLUDE_DIRS
XXX_YYY_DEFINITIONS
XXX_ZZZ_LIBRARIES
XXX_ZZZ_INCLUDE_DIRS
XXX_ZZZ_DEFINITIONS

Let's say that component YYY has been found, but ZZZ is not, so the
XXX_YYY_* variables are set, as well as the XXX_{LIBRARIES,INCLUDE_DIRS,
DEFINITIONS} variables. 

So regardless of the REQUIRED flag you end up with these variable-values:
XXX_FOUND = TRUE
XXX_YYY_FOUND = TRUE
XXX_ZZZ_FOUND = FALSE (or XXX_ZZZ-NOTFOUND, whatever you like most)

A fatal error is raised, if REQUIRED is true and XXX_FIND_REQUIRED_ZZZ is true:

find_package(XXX REQUIRED YYY) 
 -> OK, user can trust that XXX_FOUND and XXX_YYY_FOUND are true.

find_package(XXX COMPONENTS ZZZ)
 -> OK, user has to check XXX_FOUND and XXX_ZZZ_FOUND
find_package(XXX REQUIRED ZZZ)
 -> FATAL_ERROR, cmake will abort, so XXX_FOUND=true doesn't matter.

Is this an interface that you could use? Any criticism?

>>> - Interpretation of XXX_FOUND: Config files can't set the value of this
>>> variable, but find modules can, so one should think about what it means
>>> for XXX_FOUND if a component - requested or not - hasn't been found.
>> 
>> As I have written above, I don't think that the components should alter
>> the values of the package-wide variables (XXX_LIBRARIES etc.). The same
>> applies to the XXX_FOUND variable. If you search for library XXX and 
>> component YYY, XXX is still found even if it lacks the requested component.
>> If you want to know if XXX was found, you use XXX_FOUND, if you want the 
>> same for component YYY, you use XXX_YYY_FOUND.
> 
> While this interpretation of XXX_FOUND is absolutely right, IMO, the
> intention of XXX_LIBRARIES etc. is different as I said before: These
> variables should comprehend all stuff necessary to use all requested
> components. E.g., suppose YYY needs zlib; where are you going to put
> ZLIB_LIBRARY from the FIND_PACKAGE(ZLIB) invocation? XXX_YYY_LIBRARY
> is reserved for libXXX_YYY.so or the like only. It's XXX_LIBRARIES
> that takes XXX_YYY_LIBRARY along with ZLIB_LIBRARY and the other
> components' libraries and prerequisites, so

Just like XXX_LIBRARIES for package XXX, you should have XXX_YYY_LIBRARIES
for component YYY. XXX_YYY_LIBRARIES contains both XXX_YYY_LIBARY and
ZLIB_LIBRARIES. A user of the package XXX should never directly use 
XXX_YYY_LIBRARY, just like he or she should never use XXX_LIBRARY.

> ADD_DEFINITIONS(${XXX_DEFINITIONS})
> INCLUDE_DIRECTORIES(${XXX_INCLUDE_DIRS})
> TARGET_LINK_LIBRARIES(... ${XXX_LIBRARIES})
> 
> is basically sufficient to prepare for the use of the componentized XXX.

Apart from the overlinking-issues I described above...

>>> None of these questions arises for single-component packages, and while
>>> there are reasonable answers to all of them, IMO, CMake's accompanying
>>> documentation could perhaps be enhanced a bit w.r.t. multi-component
>>> packages; particularly, a simple but yet complete example how to do
>>> things would be quite helpful. 
>> 
>> I definitely agree. Multi-component packages (and package-finding in 
>> general) are often a complex matter, and the documentation on this topic
>> is not easy to read.
> 
> That's why it's very good to have another discussion about this topic. :)
> 
Indeed. ;-)

Cheers,
  Johannes