[CMake] Poor performance of copy_if_different

[Robert Dailey]

I figured out the problem.

On single-configuration generators (like NMake, which I'm testing now)
both release and debug configurations have the same binary output
directory. Since I'm creating 1 custom target (and corresponding
commands) per configuration, the 2nd configuration has the same
outputs as the 1st, so the 2nd target fails because I"m adding
different rules to the same output file. This isn't a problem with
Visual Studio generators because my outputs go to Debug and Release.

Know any good workarounds for this?

On Tue, Nov 6, 2012 at 11:54 AM, Clinton Stimpson <clinton at elemtech.com> wrote:
>
> Try this:
>
>     add_custom_command(
>         OUTPUT ${bin_outputs}
>         COMMAND ${copy_commands}
>         DEPENDS ....
>     )
>
> On Tuesday, November 06, 2012 11:50:54 AM Robert Dailey wrote:
>> Thanks Clinton,
>>
>> Here is what I have so far:
>>
>>
>> project( copy_dlls )
>>
>> set( copycmd "${CMAKE_COMMAND}" ARGS "-E" "copy_if_different" )
>>
>> set( configurations debug release )
>> foreach( config ${configurations} )
>>     string( TOUPPER ${config} upper_config )
>>
>>     get_property( binarydirs GLOBAL PROPERTY
>> THIRD_PARTY_DIRS_${upper_config} ) foreach( dir ${binarydirs} )
>>         file( GLOB binaries "${dir}/*${CMAKE_SHARED_LIBRARY_SUFFIX}" )
>>
>>         if( WIN32 AND config STREQUAL "debug" )
>>             file( GLOB pdbs "${dir}/*.pdb" )
>>             list( APPEND binaries ${pdbs} )
>>         endif()
>>
>>         foreach( bin ${binaries} )
>>             get_filename_component( bin_file ${bin} NAME )
>>
>>             list( APPEND bin_outputs ${bin} )
>>             list( APPEND copy_commands
>>                 COMMAND ${copycmd} "${bin}"
>> "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/${CMAKE_CFG_INTDIR}/${bin_file}"
>>             )
>>         endforeach()
>>     endforeach()
>>
>>     add_custom_command(
>>         OUTPUT ${bin_outputs}
>>         ${copy_commands}
>>     )
>>
>>     add_custom_target( copy_dlls_${config}
>>         COMMENT "Copying ${config} binaries..."
>>         DEPENDS ${bin_outputs}
>>     )
>> endforeach()
>>
>>
>> This doesn't seem to work though, CMake tells me:
>>
>> CMake Error: Attempt to add a custom rule to output
>> "C:/Work/rdailey-hp/dpd-cmake/build-nmake-vc9/third_party/bdb/4.7.25/vc9sp1/
>> debug/bin/libdb47d.dll.rule" which already has a custom rule.
>>
>> I'm trying to stuff all the copy commands and output files into 1
>> custom command, and wrap that with a custom target, so I can make
>> other targets depend on the custom target.
>>
>> On Tue, Nov 6, 2012 at 11:44 AM, Clinton Stimpson <clinton at elemtech.com>
> wrote:
>> > add_custom_command(
>> >
>> >   OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/my.dll
>> >   COMMAND cmake -E copy ${CMAKE_CURRENT_SOURCE_DIR}/my.dll
>> >
>> > ${CMAKE_CURRENT_BINARY_DIR}/my.dll
>> >
>> >   DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/my.dll
>> >   )
>> >
>> > add_custom_target(copy_dll ALL DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/my.dll)
>> >
>> > Whether you use 1 custom command per dll, or 1 for all dlls, is up to you.
>> >  If they usually change together, then you could have fewer custom
>> > commands.  In that case, the OUTPUT would have multiple files and both
>> > DEPENDS would have multiple files.
>> >
>> > Another alternative is to use add_custom_command(TARGET POST_BUILD ...) if
>> > these are dlls that you compile yourself and need to copy after the build
>> > step.
>> >
>> > On Tuesday, November 06, 2012 11:23:03 AM Robert Dailey wrote:
>> >> Also, is it safe to add 1 custom command for each DLL copy? Or would
>> >> you somehow only use 1 custom command and copy them all with it? Also
>> >> would I add the custom commands to the custom target?
>> >>
>> >> On Tue, Nov 6, 2012 at 11:11 AM, Clinton Stimpson <clinton at elemtech.com>
>> >
>> > wrote:
>> >> > On Tuesday, November 06, 2012 11:06:45 AM Robert Dailey wrote:
>> >> >> I use ${CMAKE_COMMAND} -E copy_if_different to copy DLL files to my
>> >> >> binary output directory. The custom target runs this command about
>> >> >> 50-100 times (for that many files).
>> >> >>
>> >> >> I notice that when all files are already copied, the commands still
>> >> >> run extremely slowly. It takes just as long to copy all the files as
>> >> >> it does to copy none of them.
>> >> >>
>> >> >> I used the Sysinternals tool called Process Monitor to see what CMake
>> >> >> is doing as it runs copy_if_different. It's opening the full file and
>> >> >> seems to be comparing actual contents instead of something simple,
>> >> >> such as timestamp.
>> >> >>
>> >> >> I do not need such a thorough check, I simply want the check to see
>> >> >> which timestamp is higher and copy if the source is newer than the
>> >> >> target.
>> >> >>
>> >> >> Any reason why copy_if_different is so slow? Is my assumption correct?
>> >> >> How can I make it faster?
>> >> >
>> >> > How about using plain "cmake -E copy ..." and rely on the timestamp
>> >> > check
>> >> > done by a custom command (add_custom_command()).
>> >> > You need to make sure the input/ouput parts of the custom command are
>> >> > set
>> >> > correctly so it can do a timestamp check.
>> >> >
>> >> > --
>> >> > Clinton Stimpson
>> >> > Elemental Technologies, Inc
>> >> > Computational Simulation Software, LLC
>> >> > www.csimsoft.com
>> >> > --
>> >> >
>> >> > Powered by www.kitware.com
>> >> >
>> >> > Visit other Kitware open-source projects at
>> >> > http://www.kitware.com/opensource/opensource.html
>> >> >
>> >> > Please keep messages on-topic and check the CMake FAQ at:
>> >> > http://www.cmake.org/Wiki/CMake_FAQ
>> >> >
>> >> > Follow this link to subscribe/unsubscribe:
>> >> > http://www.cmake.org/mailman/listinfo/cmake
>> >
>> > --
>> > Clinton Stimpson
>> > Elemental Technologies, Inc
>> > Computational Simulation Software, LLC
>> > www.csimsoft.com
>> > --
>> >
>> > Powered by www.kitware.com
>> >
>> > Visit other Kitware open-source projects at
>> > http://www.kitware.com/opensource/opensource.html
>> >
>> > Please keep messages on-topic and check the CMake FAQ at:
>> > http://www.cmake.org/Wiki/CMake_FAQ
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://www.cmake.org/mailman/listinfo/cmake
> --
> Clinton Stimpson
> Elemental Technologies, Inc
> Computational Simulation Software, LLC
> www.csimsoft.com
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake