[CMake] Poor performance of copy_if_different
Clinton Stimpson
clinton at elemtech.com
Tue Nov 6 14:07:41 EST 2012
On Tuesday, November 06, 2012 12:46:01 PM Robert Dailey wrote:
> Is there an assumption here that CMAKE_CONFIGURATION_TYPES will not be
> set for single-configuration generators?
Yes, that's the assumption. That's how I normally check for single vs. multi
config generators.
>
> I have some code in my root CMake script that forces only 'debug' and
> 'release' configurations to exist (I didn't want the MinRelSize and
> friends in there). I think this will force the logic that depends on
> CMAKE_CONFIGURATION_TYPES to fail. What would you do in this case?
>
> Here is the code that sets CMAKE_CONFIGURATION_TYPES:
>
> cmake_minimum_required( VERSION 2.8.8 FATAL_ERROR )
>
> set( CMAKE_BUILD_TYPE debug )
> set( CMAKE_CONFIGURATION_TYPES debug release CACHE INTERNAL "" FORCE )
>
> project( gmms )
>
> ... etc ...
It doesn't make much sense to me why one could only do a release build with a
mulit-config generator... I normally leave those variables alone and let the
user set them. I find RelWithDebInfo quite useful sometimes.
But if you had to, perhaps you could do this, after your project(gmms):
if(CMAKE_CONFIGURATION_TYPES)
set( CMAKE_CONFIGURATION_TYPES debug release CACHE INTERNAL "" FORCE )
endif()
Or some code to strip out types from CMAKE_CONFIGURATION_TYPES except for
debug and release. So if it was empty to begin with, it could stay empty.
>
> On Tue, Nov 6, 2012 at 12:41 PM, Clinton Stimpson <clinton at elemtech.com>
wrote:
> > You could base your logic on CMAKE_CONFIGURATION_TYPES and CMAKE_BUILD
> > TYPE.
> >
> > For example:
> >
> > if(CMAKE_CONFIGURATION_TYPES)
> >
> > foreach(type ${CMAKE_CONFIGURATION_TYPES})
> >
> > make_copy_rules(${type})
> >
> > endforeach()
> >
> > else()
> >
> > make_copy_rules(${CMAKE_BUILD_TYPE})
> >
> > endif()
> >
> > On Tuesday, November 06, 2012 12:34:22 PM Robert Dailey wrote:
> >> I figured out the problem.
> >>
> >> On single-configuration generators (like NMake, which I'm testing now)
> >> both release and debug configurations have the same binary output
> >> directory. Since I'm creating 1 custom target (and corresponding
> >> commands) per configuration, the 2nd configuration has the same
> >> outputs as the 1st, so the 2nd target fails because I"m adding
> >> different rules to the same output file. This isn't a problem with
> >> Visual Studio generators because my outputs go to Debug and Release.
> >>
> >> Know any good workarounds for this?
> >>
> >> On Tue, Nov 6, 2012 at 11:54 AM, Clinton Stimpson <clinton at elemtech.com>
> >
> > wrote:
> >> > Try this:
> >> > add_custom_command(
> >> >
> >> > OUTPUT ${bin_outputs}
> >> > COMMAND ${copy_commands}
> >> > DEPENDS ....
> >> >
> >> > )
> >> >
> >> > On Tuesday, November 06, 2012 11:50:54 AM Robert Dailey wrote:
> >> >> Thanks Clinton,
> >> >>
> >> >> Here is what I have so far:
> >> >>
> >> >>
> >> >> project( copy_dlls )
> >> >>
> >> >> set( copycmd "${CMAKE_COMMAND}" ARGS "-E" "copy_if_different" )
> >> >>
> >> >> set( configurations debug release )
> >> >> foreach( config ${configurations} )
> >> >>
> >> >> string( TOUPPER ${config} upper_config )
> >> >>
> >> >> get_property( binarydirs GLOBAL PROPERTY
> >> >>
> >> >> THIRD_PARTY_DIRS_${upper_config} ) foreach( dir ${binarydirs} )
> >> >>
> >> >> file( GLOB binaries "${dir}/*${CMAKE_SHARED_LIBRARY_SUFFIX}" )
> >> >>
> >> >> if( WIN32 AND config STREQUAL "debug" )
> >> >>
> >> >> file( GLOB pdbs "${dir}/*.pdb" )
> >> >> list( APPEND binaries ${pdbs} )
> >> >>
> >> >> endif()
> >> >>
> >> >> foreach( bin ${binaries} )
> >> >>
> >> >> get_filename_component( bin_file ${bin} NAME )
> >> >>
> >> >> list( APPEND bin_outputs ${bin} )
> >> >> list( APPEND copy_commands
> >> >>
> >> >> COMMAND ${copycmd} "${bin}"
> >> >>
> >> >> "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/${CMAKE_CFG_INTDIR}/${bin_file}"
> >> >>
> >> >> )
> >> >>
> >> >> endforeach()
> >> >>
> >> >> endforeach()
> >> >>
> >> >> add_custom_command(
> >> >>
> >> >> OUTPUT ${bin_outputs}
> >> >> ${copy_commands}
> >> >>
> >> >> )
> >> >>
> >> >> add_custom_target( copy_dlls_${config}
> >> >>
> >> >> COMMENT "Copying ${config} binaries..."
> >> >> DEPENDS ${bin_outputs}
> >> >>
> >> >> )
> >> >>
> >> >> endforeach()
> >> >>
> >> >>
> >> >> This doesn't seem to work though, CMake tells me:
> >> >>
> >> >> CMake Error: Attempt to add a custom rule to output
> >> >> "C:/Work/rdailey-hp/dpd-cmake/build-nmake-vc9/third_party/bdb/4.7.25/v
> >> >> c9s
> >> >> p1/ debug/bin/libdb47d.dll.rule" which already has a custom rule.
> >> >>
> >> >> I'm trying to stuff all the copy commands and output files into 1
> >> >> custom command, and wrap that with a custom target, so I can make
> >> >> other targets depend on the custom target.
> >> >>
> >> >> On Tue, Nov 6, 2012 at 11:44 AM, Clinton Stimpson
> >> >> <clinton at elemtech.com>
> >> >
> >> > wrote:
> >> >> > add_custom_command(
> >> >> >
> >> >> > OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/my.dll
> >> >> > COMMAND cmake -E copy ${CMAKE_CURRENT_SOURCE_DIR}/my.dll
> >> >> >
> >> >> > ${CMAKE_CURRENT_BINARY_DIR}/my.dll
> >> >> >
> >> >> > DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/my.dll
> >> >> > )
> >> >> >
> >> >> > add_custom_target(copy_dll ALL DEPENDS
> >> >> > ${CMAKE_CURRENT_BINARY_DIR}/my.dll)
> >> >> >
> >> >> > Whether you use 1 custom command per dll, or 1 for all dlls, is up
> >> >> > to
> >> >> > you.
> >> >> >
> >> >> > If they usually change together, then you could have fewer custom
> >> >> >
> >> >> > commands. In that case, the OUTPUT would have multiple files and
> >> >> > both
> >> >> > DEPENDS would have multiple files.
> >> >> >
> >> >> > Another alternative is to use add_custom_command(TARGET POST_BUILD
> >> >> > ...)
> >> >> > if
> >> >> > these are dlls that you compile yourself and need to copy after the
> >> >> > build
> >> >> > step.
> >> >> >
> >> >> > On Tuesday, November 06, 2012 11:23:03 AM Robert Dailey wrote:
> >> >> >> Also, is it safe to add 1 custom command for each DLL copy? Or
> >> >> >> would
> >> >> >> you somehow only use 1 custom command and copy them all with it?
> >> >> >> Also
> >> >> >> would I add the custom commands to the custom target?
> >> >> >>
> >> >> >> On Tue, Nov 6, 2012 at 11:11 AM, Clinton Stimpson
> >> >> >> <clinton at elemtech.com>
> >> >> >
> >> >> > wrote:
> >> >> >> > On Tuesday, November 06, 2012 11:06:45 AM Robert Dailey wrote:
> >> >> >> >> I use ${CMAKE_COMMAND} -E copy_if_different to copy DLL files to
> >> >> >> >> my
> >> >> >> >> binary output directory. The custom target runs this command
> >> >> >> >> about
> >> >> >> >> 50-100 times (for that many files).
> >> >> >> >>
> >> >> >> >> I notice that when all files are already copied, the commands
> >> >> >> >> still
> >> >> >> >> run extremely slowly. It takes just as long to copy all the
> >> >> >> >> files
> >> >> >> >> as
> >> >> >> >> it does to copy none of them.
> >> >> >> >>
> >> >> >> >> I used the Sysinternals tool called Process Monitor to see what
> >> >> >> >> CMake
> >> >> >> >> is doing as it runs copy_if_different. It's opening the full
> >> >> >> >> file
> >> >> >> >> and
> >> >> >> >> seems to be comparing actual contents instead of something
> >> >> >> >> simple,
> >> >> >> >> such as timestamp.
> >> >> >> >>
> >> >> >> >> I do not need such a thorough check, I simply want the check to
> >> >> >> >> see
> >> >> >> >> which timestamp is higher and copy if the source is newer than
> >> >> >> >> the
> >> >> >> >> target.
> >> >> >> >>
> >> >> >> >> Any reason why copy_if_different is so slow? Is my assumption
> >> >> >> >> correct?
> >> >> >> >> How can I make it faster?
> >> >> >> >
> >> >> >> > How about using plain "cmake -E copy ..." and rely on the
> >> >> >> > timestamp
> >> >> >> > check
> >> >> >> > done by a custom command (add_custom_command()).
> >> >> >> > You need to make sure the input/ouput parts of the custom command
> >> >> >> > are
> >> >> >> > set
> >> >> >> > correctly so it can do a timestamp check.
> >> >> >> >
> >> >> >> > --
> >> >> >> > Clinton Stimpson
> >> >> >> > Elemental Technologies, Inc
> >> >> >> > Computational Simulation Software, LLC
> >> >> >> > www.csimsoft.com
> >> >> >> > --
> >> >> >> >
> >> >> >> > Powered by www.kitware.com
> >> >> >> >
> >> >> >> > Visit other Kitware open-source projects at
> >> >> >> > http://www.kitware.com/opensource/opensource.html
> >> >> >> >
> >> >> >> > Please keep messages on-topic and check the CMake FAQ at:
> >> >> >> > http://www.cmake.org/Wiki/CMake_FAQ
> >> >> >> >
> >> >> >> > Follow this link to subscribe/unsubscribe:
> >> >> >> > http://www.cmake.org/mailman/listinfo/cmake
> >> >> >
> >> >> > --
> >> >> > Clinton Stimpson
> >> >> > Elemental Technologies, Inc
> >> >> > Computational Simulation Software, LLC
> >> >> > www.csimsoft.com
> >> >> > --
> >> >> >
> >> >> > Powered by www.kitware.com
> >> >> >
> >> >> > Visit other Kitware open-source projects at
> >> >> > http://www.kitware.com/opensource/opensource.html
> >> >> >
> >> >> > Please keep messages on-topic and check the CMake FAQ at:
> >> >> > http://www.cmake.org/Wiki/CMake_FAQ
> >> >> >
> >> >> > Follow this link to subscribe/unsubscribe:
> >> >> > http://www.cmake.org/mailman/listinfo/cmake
> >> >
> >> > --
> >> > Clinton Stimpson
> >> > Elemental Technologies, Inc
> >> > Computational Simulation Software, LLC
> >> > www.csimsoft.com
> >> > --
> >> >
> >> > Powered by www.kitware.com
> >> >
> >> > Visit other Kitware open-source projects at
> >> > http://www.kitware.com/opensource/opensource.html
> >> >
> >> > Please keep messages on-topic and check the CMake FAQ at:
> >> > http://www.cmake.org/Wiki/CMake_FAQ
> >> >
> >> > Follow this link to subscribe/unsubscribe:
> >> > http://www.cmake.org/mailman/listinfo/cmake
> >
> > --
> > Clinton Stimpson
> > Elemental Technologies, Inc
> > Computational Simulation Software, LLC
> > www.csimsoft.com
> > --
> >
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the CMake FAQ at:
> > http://www.cmake.org/Wiki/CMake_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.cmake.org/mailman/listinfo/cmake
--
Clinton Stimpson
Elemental Technologies, Inc
Computational Simulation Software, LLC
www.csimsoft.com
More information about the CMake
mailing list