[CMake] Reusing an already built object

Sat Jun 12 23:30:50 EDT 2010

On 06/12/2010 04:10 AM, Linghua Tseng wrote:
> Continue the lastest post of this article:
> http://www.cmake.org/pipermail/cmake/2009-October/032615.html
> 
>> On Tuesday 13 October 2009, Alexander Neundorf wrote:
>>> On Tuesday 13 October 2009, Naram Qashat wrote:
>>> Say I have a main executable and a number of shared libraries that rely on
>>> that executable.  Say I have a certain C++ source that is required to be
>>> built by not only the main executable, but also every single shared library
>>> (the source in question is a Windows-specific file to handle memory
>>> allocations).  I have found that when using a Visual Studio generator,
>>> CMake causes the source file to be rebuilt for every single shared library,
>>> causing an increase in build time.  Is there an easy way to have the object
>>> file not be rebuilt every single time, but be reused for all the shared
>>> libraries?  
>>
>> No, it's a feature.
>> You may use different compile flags etc. for your different targets, so you 
>> may get different object files.
>>
>> You may hack around that by creating a static library, figuring out the path 
>> to the object files and then link these object files directly to your targets 
>> (or something like that, but I've never done that and I don't recommend it).
> 
> Consider the following scenario:
> 
> gcc -c src1.c
> gcc gen_src2.c -o gen_src2 src1.o
> ./gen_src2 > src2.c
> 
> gcc -c src2.c
> gcc gen_src3.c -o gen_src3 src1.o src2.o
> ./gen_src3 > src3.c
> 
> gcc -c src3.c
> gcc gen_src4.c -o gen_src4 src1.o src2.o src3.o
> ./gen_src4 > src4.c
> 
> ...
> 
> gcc -c src99.c
> gcc gen_src100.c -o gen_src100 src1.o src2.o src3.o ... src99.o
> ./gen_src100 > src100.c
> 
> gcc -c src100.c
> ar rcu libmylib.a src1.o src2.o src3.o ... src100.o
> 
> 
> To create a project by using CMake,
> we'll have several targets by using add_executable():
>   gen_src2, gen_src3, gen_src4, ... gen_src100
> And we'll have a target by using add_library():
>   mylib
> 
> Assume we already have gen_src1.c, gen_src2.c, ..., gen_src100.c, and src1.c
> Without lost of generality, I reduce the example to src1 ~ src4.
> Then we have to write the following CMakeLists.txt:
> 
> project(main)
> cmake_minimum_required(VERSION 2.8)
> 
> set(gen_src2_SRCS src1.c gen_src2.c)
> add_executable(gen_src2 ${gen_src2_SRCS})
> get_target_property(gen_src2_EXE gen_src2 LOCATION)
> add_custom_command(
>         OUTPUT src2.c
>         COMMAND ${gen_src2_EXE}
>         ARGS > src2.c
>         DEPENDS gen_src2
> )
> 
> set(gen_src3_SRCS src1.c ${PROJECT_BINARY_DIR}/src2.c gen_src3.c)
> add_executable(gen_src3 ${gen_src3_SRCS})
> get_target_property(gen_src3_EXE gen_src3 LOCATION)
> add_custom_command(
>         OUTPUT src3.c
>         COMMAND ${gen_src3_EXE}
>         ARGS > src3.c
>         DEPENDS gen_src3
> )
> 
> set(gen_src4_SRCS src1.c ${PROJECT_BINARY_DIR}/src2.c ${PROJECT_BINARY_DIR}/src3.c gen_src4.c)
> add_executable(gen_src4 ${gen_src4_SRCS})
> get_target_property(gen_src4_EXE gen_src4 LOCATION)
> add_custom_command(
>         OUTPUT src4.c
>         COMMAND ${gen_src4_EXE}
>         ARGS > src4.c
>         DEPENDS gen_src4
> )
> 
> set(mylib_SRCS src1.c
>                ${PROJECT_BINARY_DIR}/src2.c
>                ${PROJECT_BINARY_DIR}/src3.c
>                ${PROJECT_BINARY_DIR}/src4.c
> )
> add_library(mylib ${PROJECT_BINARY_DIR}/src4.c)

Do you rather mean "add_library(mylib ${mylib_SRCS})" here?

> Now the compile-flow shows:
> [  5%] Building C object CMakeFiles/gen_src2.dir/src1.c.o
> [ 10%] Building C object CMakeFiles/gen_src2.dir/gen_src2.c.o
> Linking C executable gen_src2
> [ 10%] Built target gen_src2
> [ 15%] Generating src2.c
> [ 21%] Building C object CMakeFiles/gen_src3.dir/src1.c.o
> [ 26%] Building C object CMakeFiles/gen_src3.dir/src2.c.o
> [ 31%] Building C object CMakeFiles/gen_src3.dir/gen_src3.c.o
> Linking C executable gen_src3
> [ 31%] Built target gen_src3
> [ 36%] Generating src3.c
> [ 42%] Building C object CMakeFiles/gen_src4.dir/src1.c.o
> [ 47%] Building C object CMakeFiles/gen_src4.dir/src2.c.o
> [ 52%] Building C object CMakeFiles/gen_src4.dir/src3.c.o
> [ 57%] Building C object CMakeFiles/gen_src4.dir/gen_src4.c.o
> Linking C executable gen_src4
> [ 63%] Built target gen_src4
> [ 68%] Generating src4.c
> [ 73%] Building C object CMakeFiles/mylib.dir/src1.c.o
> [ 78%] Building C object CMakeFiles/mylib.dir/src2.c.o
> [ 84%] Building C object CMakeFiles/mylib.dir/src3.c.o
> [ 89%] Building C object CMakeFiles/mylib.dir/src4.c.o
> Linking C static library libmylib.a
> [100%] Built target mylib
> 
> In this case, `src1.c.o' was built 4 times.
> If the example is src1~src100, it will need to be compiled 100 times.
> It's really unnecessary because I don't want to use different CFLAGS for building them.

Look at the following CMakeLists.txt:

project(main)
cmake_minimum_required(VERSION 2.8)

add_library(gen1 STATIC src1.c)
set(gen_src2_SRCS gen_src2.c)
add_executable(gen_src2 ${gen_src2_SRCS})
target_link_libraries(gen_src2 gen1)
get_target_property(gen_src2_EXE gen_src2 LOCATION)
add_custom_command(
        OUTPUT src2.c
        COMMAND ${gen_src2_EXE}
        ARGS > src2.c
        DEPENDS gen_src2
)

add_library(gen2 STATIC ${PROJECT_BINARY_DIR}/src2.c)
set(gen_src3_SRCS gen_src3.c)
add_executable(gen_src3 ${gen_src3_SRCS})
target_link_libraries(gen_src3 gen2 gen1)
get_target_property(gen_src3_EXE gen_src3 LOCATION)
add_custom_command(
        OUTPUT src3.c
        COMMAND ${gen_src3_EXE}
        ARGS > src3.c
        DEPENDS gen_src3
)

add_library(gen3 STATIC ${PROJECT_BINARY_DIR}/src3.c)
set(gen_src4_SRCS gen_src4.c)
add_executable(gen_src4 ${gen_src4_SRCS})
target_link_libraries(gen_src4 gen3 gen2 gen1)
get_target_property(gen_src4_EXE gen_src4 LOCATION)
add_custom_command(
        OUTPUT src4.c
        COMMAND ${gen_src4_EXE}
        ARGS > src4.c
        DEPENDS gen_src4
)

set(mylib_SRCS src1.c
               ${PROJECT_BINARY_DIR}/src2.c
               ${PROJECT_BINARY_DIR}/src3.c
               ${PROJECT_BINARY_DIR}/src4.c
)
add_library(mylib ${mylib_SRCS})

After cmaking, a "make | grep Building" yields:

[  6%] Building C object CMakeFiles/gen1.dir/src1.c.o
[ 13%] Building C object CMakeFiles/gen_src2.dir/gen_src2.c.o
[ 26%] Building C object CMakeFiles/gen2.dir/src2.c.o
[ 33%] Building C object CMakeFiles/gen_src3.dir/gen_src3.c.o
[ 46%] Building C object CMakeFiles/gen3.dir/src3.c.o
[ 53%] Building C object CMakeFiles/gen_src4.dir/gen_src4.c.o
[ 66%] Building C object CMakeFiles/mylib.dir/src1.c.o
[ 73%] Building C object CMakeFiles/mylib.dir/src2.c.o
[ 80%] Building C object CMakeFiles/mylib.dir/src3.c.o
[ 86%] Building C object CMakeFiles/mylib.dir/src4.c.o

Thus, the sources whose object files will be incorporated in the
executables as well as in your library are compiled just twice, and this
is unavoidable, or at least shouldn't be bypassed, as AN has pointed out.

> I know someone said I can build static libraries for avoiding this,
> but it will fall into another issue:
> [Cmake] How do I link a static library into a library
> http://www.cmake.org/pipermail/cmake/2004-April/004990.html
> Therefore, it still cannot solve my problem.

If I understand correctly, the concern of that thread's OP was to
enhance a static library with another one that was built externally,
i.e. outside the project. In your project, you can decide when and from
which object files a static library is built, so you don't need to stick
to a single library, in particular. Instead, you can adapt to your, say,
incremental build process and generate one static library per step which
will be used in later steps in order to avoid numerous recompilations of
the same source files.

'hope that helps.

Regards,

Michael