[CMake] custom target isn't rebuilt if depending on another custom target

[Michael Hertling]

On 09/14/2010 11:56 AM, Gerhard Stengel wrote:
> Hello Michael,
> 
> thanks, it sheds some light on the matter, but I'm not happy as it is, see comments below.
> 
> regards
> 
> Gerhard
> ...
>>>
>>> If I create Demo.tar from a clean project, everything's fine. However,
>>> the rebuilt isn't performed completely, that is if I touch Demo1.in,
>>> just Demo1.out is rebuilt, but not Demo.tar that depends on it! [...]
>>
>> Demo.tar does *not* depend on Demo1.out; it has no dependencies at all.
>> As a proof - if on *nix - issue "find . -exec grep "Demo\.tar" {} \;"
>> in the build directory after cmaking, and you'll see that Demo.tar's
>> dependency line doesn't mention any prerequisites after the colon.
> Well, I can confirm this, but I actually expected to see targets like cust1 and cust2 here.

If you look at the dependency graph you'll see that there is no path
from Demo.tar to cust{1,2}, so which command in your CMakeLists.txt
is meant to establish these dependencies?

> (Maybe this is a naiive notion and it's not possible to list targets as dependencies, I'm no expert to make ;-)

It is possible, but probably doesn't work as you desire with custom targets:

ADD_CUSTOM_COMMAND(OUTPUT <file> ... COMMAND ... DEPENDS <target> ...)

with <target> being a custom target from another CMakeLists.txt
results in rebuilding <target> before <file> is used for a further
target, but the COMMAND won't be run just because <target> is rebuilt.

>>> [...] This seems wrong to me because I
>>> expect that if the target cust1 is rebuild, even the target cust3 has to
>>> be rebuilt due to the relation established by the add_dependencies()
>>> command!
>>
>> Rebuilding cust3 doesn't regenerate Demo.tar due to its lack of
>> dependencies if it already exists. So, Demo.tar is generated at
>> the first time and that's it, unless the file is removed later.
> But what is then the purpose of the add_dependencies command if it only works the *first* time and never again?

ADD_DEPENDENCIES() works all the time and cust3 is actually rebuilt,
look at make's output. The problem is that rebuilding cust3 doesn't
regenerate Demo.tar because there is no reason to do so: Updating a
target means checking if its prerequisites and, recursively, their
prerequisites are up to date, rebuild those ones being out of date
and, finally, rebuild the target if at least one prerequisite has
been rebuilt. For cust3, the prerequisite is, apart from cust{1,2},
Demo.tar, and if this file exists and cust{1,2} are rebuilt all of
cust3's dependencies are satisfied, no commands are associated, so
nothing happens.

> For me the term "cust3 depends on cust1" means two things when make checks cust3:
> 1) if cust1 (and all associated files) are not built yet, they shall be generated. Then cust3 is generated.
> 2) if cust1 has been updated, cust3 is re-generated

Right, absolutely.

> If add_dependencies() only covers case #1, it seems to me quite useless.

It also covers case #2; it's the gap in the dependency graph, i.e. the
missing link Demo.tar --> Demo{1,2}.out, that's causing the undesired
behaviour.

>>> The problem seems to be that add_dependencies() doesn't realize that if
>>> cust1 or cust2 becomes out of date cust3 has to be rebuilt, too.
>>
>> It does, but rebuilding cust3 doesn't do what you expect, see above.
>>
>>> The solution using file level dependencies (see commented out command)
>>> would work, [...]
>>
>> ...and is the correct one...
>>
>>> [...] but in my project it's not really
>>> possible to do so because the "source" custom targets which the "final"
>>> custom target takes as input contain lots of files.
>>
>> First of all, is it possible to simplify your CMakeLists.txt? Unless
>> you really need cust{1,2} as top-level targets they aren't necessary
>> if Demo{1,2}.out are mentioned as dependencies in Demo.tar's custom
>> command and if these commands all reside in the same CMakeLists.txt.
>> Furthermore, you could even use
>>
>> ADD_CUSTOM_TARGET(Demo.tar ALL
>>   COMMAND ${CMAKE_COMMAND} -E tar cvf Demo.tar Demo1.out Demo2.out
>>   DEPENDS Demo1.out Demo2.out)
>>
>> instead of Demo.tar's custom command if you don't need Demo.tar as an
>> ingredient for another target and, again, if these commands/targets
>> are defined in the same CMakeLists.txt.
>>
> Well, the problem is in fact that each custom target and the associated files reside in an own folder, and some of these 
> custom targets resp. files depend on each other. To be more precise, when the *.out files are generated from the *.in 
> files, the generator needs some other *.out files (from different folders). The *.in files include the *.out files and 
> the generator tool reads this information. 

OK, so, you would need either file-level dependencies across several
CMakeLists.txt or custom targets which trigger custom commands when
mentioned among their prerequisites. AFAIK, both approaches don't
work :( but perhaps, there're acceptable alternatives, see below.

> My idea was to "group" each folders *.out files into an own custom target. If the generation process needs certain files 
> from a certain folder, I would just establish a target level dependency with add_dependencies() to this folder's custom 
> target.

Probably, this doesn't work due to the above-mentioned reasons, i.e.
without a DEPENDS clause, a custom command won't be run, and custom
targets as well as OUTPUT files from other CMakeLists.txt don't suit
as dependencies to run the command. What you could do alternatively:

1) Use custom targets instead of custom commands, but this would
   trigger your complete generation process each time you run make.
2) Reorganize your project's build configuration in such a way that
   all custom targets/commands related to the generation process are
   concentrated in a single CMakeLists.txt, so file-level dependencies
   will work. The generation's input/output files could still reside in
   multiple directories as you can use relative paths, and perhaps, you
   can make use of the INCLUDE() command to decompose the CMakeLists.txt.

Personally, I'd prefer the second approach.

> The beauty of this solution is that these targets are known globally. To exchange lists of files between folders in a 
> tree structure becomes a pain with cmakes handling of variable scopes :-/ 
> The only solution I can think of right now would be to use SET(VAR ... PARENT_SCOPE). Is there anything better?

Perhaps, properties could do a better job than variables in this regard,
but a solution with custom targets/commands and cleanly set up file-
level dependencies is the straightest and least annoying one, IMO.

Regards,

Michael