[CMake] Reusing an already built object

Linghua Tseng uranus at tinlans.org
Sun Jun 13 16:08:50 EDT 2010


On 06/12/2010 23:30:50 Michael Hertling wrote:
> On 06/12/2010 04:10 AM, Linghua Tseng wrote:
>> ...
>> set(mylib_SRCS src1.c
>>                ${PROJECT_BINARY_DIR}/src2.c
>>                ${PROJECT_BINARY_DIR}/src3.c
>>                ${PROJECT_BINARY_DIR}/src4.c
>> )
>> add_library(mylib ${PROJECT_BINARY_DIR}/src4.c)
> 
> Do you rather mean "add_library(mylib ${mylib_SRCS})" here?
Yes.

> 
> Look at the following CMakeLists.txt:
> 
> project(main)
> cmake_minimum_required(VERSION 2.8)
> 
> add_library(gen1 STATIC src1.c)
> set(gen_src2_SRCS gen_src2.c)
> add_executable(gen_src2 ${gen_src2_SRCS})
> target_link_libraries(gen_src2 gen1)
> get_target_property(gen_src2_EXE gen_src2 LOCATION)
> add_custom_command(
>        OUTPUT src2.c
>        COMMAND ${gen_src2_EXE}
>        ARGS > src2.c
>        DEPENDS gen_src2
> )
> 
> add_library(gen2 STATIC ${PROJECT_BINARY_DIR}/src2.c)
> set(gen_src3_SRCS gen_src3.c)
> add_executable(gen_src3 ${gen_src3_SRCS})
> target_link_libraries(gen_src3 gen2 gen1)
> get_target_property(gen_src3_EXE gen_src3 LOCATION)
> add_custom_command(
>        OUTPUT src3.c
>        COMMAND ${gen_src3_EXE}
>        ARGS > src3.c
>        DEPENDS gen_src3
> )
> 
> add_library(gen3 STATIC ${PROJECT_BINARY_DIR}/src3.c)
> set(gen_src4_SRCS gen_src4.c)
> add_executable(gen_src4 ${gen_src4_SRCS})
> target_link_libraries(gen_src4 gen3 gen2 gen1)
> get_target_property(gen_src4_EXE gen_src4 LOCATION)
> add_custom_command(
>        OUTPUT src4.c
>        COMMAND ${gen_src4_EXE}
>        ARGS > src4.c
>        DEPENDS gen_src4
> )
> 
> set(mylib_SRCS src1.c
>               ${PROJECT_BINARY_DIR}/src2.c
>               ${PROJECT_BINARY_DIR}/src3.c
>               ${PROJECT_BINARY_DIR}/src4.c
> )
> add_library(mylib ${mylib_SRCS})
> 
> After cmaking, a "make | grep Building" yields:
> 
> [  6%] Building C object CMakeFiles/gen1.dir/src1.c.o
> [ 13%] Building C object CMakeFiles/gen_src2.dir/gen_src2.c.o
> [ 26%] Building C object CMakeFiles/gen2.dir/src2.c.o
> [ 33%] Building C object CMakeFiles/gen_src3.dir/gen_src3.c.o
> [ 46%] Building C object CMakeFiles/gen3.dir/src3.c.o
> [ 53%] Building C object CMakeFiles/gen_src4.dir/gen_src4.c.o
> [ 66%] Building C object CMakeFiles/mylib.dir/src1.c.o
> [ 73%] Building C object CMakeFiles/mylib.dir/src2.c.o
> [ 80%] Building C object CMakeFiles/mylib.dir/src3.c.o
> [ 86%] Building C object CMakeFiles/mylib.dir/src4.c.o
> 
> Thus, the sources whose object files will be incorporated in the
> executables as well as in your library are compiled just twice, and this
> is unavoidable, or at least shouldn't be bypassed, as AN has pointed out.

I knew this approach, and I also found that I can write this line in the end of CMakeLists.txt:
  add_library(mylib gen4 gen3 gen2 gen1)
instead of re-listing sources.
(Assume that you wrote: add_library(gen4 STATIC ...) before)

Now I modified something in order to explain the next issue:
project(main)
cmake_minimum_required(VERSION 2.8)

add_library(src1 STATIC src1.c)
set(lower_layer_lib_LIBRARIES src1)

set(gen_src2_SRCS gen_src2.c)
add_executable(gen_src2 ${gen_src2_SRCS})
target_link_libraries(gen_src2 src1)
get_target_property(gen_src2_EXE gen_src2 LOCATION)
add_custom_command(
        OUTPUT src2.c
        COMMAND ${gen_src2_EXE}
        ARGS > src2.c
        DEPENDS gen_src2
)
add_library(src2 STATIC src2.c)
set(lower_layer_lib_LIBRARIES src2 ${lower_layer_lib_LIBRARIES})

set(gen_src3_SRCS gen_src3.c)
add_executable(gen_src3 ${gen_src3_SRCS})
target_link_libraries(gen_src3 src2 src1)
get_target_property(gen_src3_EXE gen_src3 LOCATION)
add_custom_command(
        OUTPUT src3.c
        COMMAND ${gen_src3_EXE}
        ARGS > src3.c
        DEPENDS gen_src3
)
add_library(src3 STATIC src3.c)
set(lower_layer_lib_LIBRARIES src3 ${lower_layer_lib_LIBRARIES})

set(gen_src4_SRCS gen_src4.c)
add_executable(gen_src4 ${gen_src4_SRCS})
target_link_libraries(gen_src4 src3 src2 src1)
get_target_property(gen_src4_EXE gen_src4 LOCATION)
add_custom_command(
        OUTPUT src4.c
        COMMAND ${gen_src4_EXE}
        ARGS > src4.c
        DEPENDS gen_src4
)
add_library(src4 STATIC src4.c)
set(lower_layer_lib_LIBRARIES src4 ${lower_layer_lib_LIBRARIES})

# Yes, it works.
add_library(lower_layer_lib ${lower_layer_lib_LIBRARIES})

>> I know someone said I can build static libraries for avoiding this,
>> but it will fall into another issue:
>> [Cmake] How do I link a static library into a library
>> http://www.cmake.org/pipermail/cmake/2004-April/004990.html
>> Therefore, it still cannot solve my problem.
> 
> If I understand correctly, the concern of that thread's OP was to
> enhance a static library with another one that was built externally,
> i.e. outside the project. In your project, you can decide when and from
> which object files a static library is built, so you don't need to stick
> to a single library, in particular. Instead, you can adapt to your, say,
> incremental build process and generate one static library per step which
> will be used in later steps in order to avoid numerous recompilations of
> the same source files.

In my simplified example, the above approach is really good.
But it's bad in the multi-layer building architecture.
Assume that libhigher_layer_lib.a contains all objects of liblower_layer_lib1.a liblower_layer_lib2.a, ..., and so on.
Of course, I don't need to build liblower_layer_lib###.a if CMake can link these object files directly.
In the following example, I reduce them to a file `liblower_layer_lib.a'.

For the multi-layer case, I appended these lines to CMakeLists.txt:
  set(lower_layer_LIBRARIES lower_layer_lib)
  set(higher_layer_lib_SRCS higher_src1.c)
  add_library(higher_layer_lib STATIC ${higher_layer_lib_SRCS} ${lower_layer_LIBRARIES})
And then I got these error messages:
  CMake Error in CMakeLists.txt:
    Cannot find source file "lower_layer_lib".  Tried extensions .c .C .c++ .cc
    .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx

To replace them by the 4 lines "seems" to work:
  set(lower_layer_LIBRARIES lower_layer_lib)
  set(higher_layer_lib_SRCS higher_src1.c)
  add_library(higher_layer_lib STATIC ${higher_layer_lib_SRCS})                         
  target_link_libraries(higher_layer_lib ${lower_layer_LIBRARIES})      
It yields:
  Scanning dependencies of target higher_layer_lib
  [ 88%] Building C object CMakeFiles/higher_layer_lib.dir/higher_src1.c.o
  Linking C static library libhigher_layer_lib.a
  [ 88%] Built target higher_layer_lib
But the real commands are:
  /usr/bin/gcc    -o CMakeFiles/higher_layer_lib.dir/higher_src1.c.o   -c /home/uranus/lab/test/cmake2/higher_src1.c
  /usr/bin/ar cr libhigher_layer_lib.a  CMakeFiles/higher_layer_lib.dir/higher_src1.c.o
  /usr/bin/ranlib libhigher_layer_lib.a

Yes, libhigher_layer_lib.a only contains higher_src1.c.o.
It doesn't contain any objects in liblower_layer_lib.a.

OK. I know that I can re-list the sources into add_library(higher_layer_lib STATIC ...).
I can also layering the sources by using a lot of SET commands.
But we have to re-compile everything in each higher layer.

Look at the following tree:
  layer1/ (this layer is required to re-compile 200 files)
    layer2a/ (this layer is required to re-compile 100 files)
      layer3a/ (50 files)
      layer3b/ (50 files)
    layer2b/ (this layer is required to re-compile 100 files)
      layer3c/ (50 files)
      layer3d/ (50 files)

Note that we have to build executable files in each layer,
and these executable files need to link the objects in its own layer and sub-layers.
Thus, we MUST build liblayer3a.a, liblayer3b.a, liblayer3c.a, liblayer3d.a, liblayer2a.c, liblayer2b.a, and ilblayer1.a.
But GNU make can just link these objects directly, so it doesn't need to build these static libraries.

If we have to spend 30 minutes to build this kind of source tree by using GNU make,
we have to spend 30 * 3 = 90 minutes to build it by using CMake.
I think it's really not reasonable.
 


More information about the CMake mailing list