MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0015916CMakeCMakepublic2016-01-13 14:172016-06-10 14:21
ReporterR Jones 
Assigned ToBrad King 
PrioritynormalSeveritytweakReproducibilityalways
StatusclosedResolutionfixed 
PlatformSunOSOSSolarisOS Version10
Product VersionCMake 3.3.1 
Target VersionCMake 3.5Fixed in VersionCMake 3.5 
Summary0015916: [Buildroot] [PATCH] cmake: use system libzma when the xz package is enabled
DescriptionI have XZ installed already.
I was trying to compile cmake using gmake and GCC4.

The error is:
Fix build failure:
.../output/build/host-cmake-3.1.3/Utilities/cmliblzma/liblzma/common/block_buffer_decoder.c:17:1: error: conflicting types for ‘lzma_block_buffer_decode’
 lzma_block_buffer_decode(lzma_block *block, lzma_allocator *allocator,
 ^
In file included from /home/tetsuya/buildroot/br2/output/host/usr/include/lzma.h:296:0,
                 from /home/tetsuya/buildroot/br2/output/build/host-cmake-3.1.3/Utilities/cmliblzma/liblzma/common/common.h:34,
                 from /home/tetsuya/buildroot/br2/output/build/host-cmake-3.1.3/Utilities/cmliblzma/liblzma/common/block_decoder.h:16,
                 from /home/tetsuya/buildroot/br2/output/build/host-cmake-3.1.3/Utilities/cmliblzma/liblzma/common/block_buffer_decoder.c:13:
/home/tetsuya/buildroot/br2/output/host/usr/include/lzma/block.h:577:27: note: previous declaration of ‘lzma_block_buffer_decode’ was here
 extern LZMA_API(lzma_ret) lzma_block_buffer_decode(
                           ^
Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/build.make:261: recipe for target 'Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/common/block_buffer_decoder.c.o' failed
make[3]: *** [Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/common/block_buffer_decoder.c.o] Error 1
Steps To ReproduceOn a Solaris system with with GCC4, GMAKE, ZLIB, EXPAT, BZIP2, CURL, XZ and without cmake installed:

CFLAGS=-I{INSTARGET}/include

bash ./configure \
            --prefix=${INSTARGET} \
            --system-zlib \
            --system-expat \
            --system-bzip2 \
            --system-curl

Causes the problem in the description.

After patching bootstrap (see below), I was able to clear the problem with:
bash ./configure \
            --prefix=${INSTARGET} \
            --system-zlib \
            --system-expat \
            --system-bzip2 \
            --system-liblzma \
            --system-curl
Additional InformationI found this thread when looking into how to disable cmake build from trying to include it's own liblzma instead of the system liblzma. It seems that the engine is prepared for it, but bootstrap does not allow --system-liblzma as an option. All I needed to do to disable the internal copy was to add this to bootstrap and re-run configure:

$ diff bootstrap bootstrap.orig
652c652
< --system-bzip2|--system-curl|--system-expat|--system-jsoncpp|--system-libarchive|--system-zlib|--system-liblzma)
---
> --system-bzip2|--system-curl|--system-expat|--system-jsoncpp|--system-libarchive|--system-zlib)

Also would need to update the "help"

Is there some reason this option could not be added to the package?
TagsNo tags attached.
Relationships
related to 0015920closed  Tests fail 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2016-01-13 14:17R JonesNew Issue
2016-01-13 14:20R JonesNote Added: 0040223
2016-01-13 14:31Brad KingNote Added: 0040225
2016-01-13 14:32Brad KingAssigned To => Brad King
2016-01-13 14:32Brad KingStatusnew => resolved
2016-01-13 14:32Brad KingResolutionopen => fixed
2016-01-13 14:32Brad KingFixed in Version => CMake 3.5
2016-01-13 14:32Brad KingTarget Version => CMake 3.5
2016-01-13 14:41Brad KingNote Added: 0040226
2016-01-13 18:03R JonesNote Added: 0040227
2016-01-14 09:14Brad KingNote Added: 0040230
2016-01-14 09:42R JonesNote Added: 0040232
2016-01-14 09:49R JonesNote Added: 0040233
2016-01-14 10:33R JonesNote Added: 0040235
2016-01-14 10:34R JonesNote Added: 0040236
2016-01-14 10:48Brad KingRelationship addedrelated to 0015920
2016-01-14 11:01Brad KingNote Added: 0040239
2016-06-10 14:21Kitware RobotNote Added: 0041290
2016-06-10 14:21Kitware RobotStatusresolved => closed

Notes
(0040223)
R Jones   
2016-01-13 14:20   
Note the example in the description is a cut/paste from the same issue on the Buildroot list at this URL:

http://lists.busybox.net/pipermail/buildroot/2015-March/120781.html [^]

The solution in that thread didn't seem to cleanly address this...such as adding the option I presented here did. So posting here.
(0040225)
Brad King   
2016-01-13 14:31   
Thanks for pointing this out. I've added the option:

 bootstrap: Add option to build with system liblzma
 https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=7a37afa7 [^]
(0040226)
Brad King   
2016-01-13 14:41   
I'd also like to figure out what caused the initial compilation error. The error message in the description shows that the system-provided "lzma.h" is somehow included by "Utilities/cmliblzma/liblzma/common/block_buffer_decoder.c" but we expect the bundled "lzma.h" to be included. This mismatch causes the error.

With "make VERBOSE=1" you should see the full compile line. Can you see why the wrong header gets included?
(0040227)
R Jones   
2016-01-13 18:03   
Looks like CFLAGS is first instead of LAST on the line.

[ 14%] Building C object Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/common/block_buffer_decoder.c.o
cd /appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma &&
  /appl/local-201601/bin/gcc
        -I/appl/local-201601/include
        -I/appl/local64-201601/include
        -w
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/common
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/api
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/check
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/common
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/delta
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/lz
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/lzma
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/rangecoder
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/simple
        -I/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma
        -std=gnu11
        -o CMakeFiles/cmliblzma.dir/liblzma/common/block_buffer_decoder.c.o
        -c /appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/common/block_buffer_decoder.c

/appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/common/block_buffer_decoder.c:17:1: error: conflicting types for 'lzma_block_buffer_decode'
In file included from /appl/local64-201601/include/lzma.h:296:0,
                 from /appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/common/common.h:34,
                 from /appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/common/block_decoder.h:16,
                 from /appl/local_build/src/cmake.org/cmake-3.3.1/Utilities/cmliblzma/liblzma/common/block_buffer_decoder.c:13:
/appl/local64-201601/include/lzma/block.h:577:27: note: previous declaration of 'lzma_block_buffer_decode' was here
Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/build.make:278: recipe for target 'Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/common/block_buffer_decoder.c.o' failed
make[2]: *** [Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/liblzma/common/block_buffer_decoder.c.o] Error 1
make[2]: Leaving directory '/appl/logs/local_build/src/cmake.org/cmake-3.3.1'
CMakeFiles/Makefile2:1430: recipe for target 'Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/all' failed
make[1]: *** [Utilities/cmliblzma/CMakeFiles/cmliblzma.dir/all] Error 2
make[1]: Leaving directory '/appl/logs/local_build/src/cmake.org/cmake-3.3.1'
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2
(0040230)
Brad King   
2016-01-14 09:14   
Re 0015916:0040227: Thanks. As of CMake 3.3 we have this template for C compilation:

 https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/CMakeCInformation.cmake;hb=v3.3.2#l187 [^]
 <CMAKE_C_COMPILER> <DEFINES> <FLAGS> -o <OBJECT> -c <SOURCE>

The <FLAGS> place holder is replaced with the CFLAGS and then the -I flags generated by CMake from include_directories commands.

As of CMake 3.4 we now have this instead:

 https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/CMakeCInformation.cmake;hb=v3.4.1#l187 [^]
 <CMAKE_C_COMPILER> <DEFINES> <INCLUDES> <FLAGS> -o <OBJECT> -c <SOURCE>

The <INCLUDES> placeholder gets the flags generated by CMake from include_directories, and the <FLAGS> placeholder still gets CFLAGS.

Therefore, this should no longer be a problem with 3.4. That is why I couldn't reproduce the issue by adding a -I to my CFLAGS.

Normally we don't expect any -I to be in CFLAGS because CMake projects find the headers they need and then use include_directories(). Why did you need to add that flag?
(0040232)
R Jones   
2016-01-14 09:42   
Because I don't want to use what's in /usr/local/include and am building up a separate /appl/local/... tree and need everything to come from there.

Are you saying that if I use "--prefix=${INSTARGET}" that ${INSTARGET}/include will be checked before /usr/include or /usr/local/include?

I have not observed this working with any tool so far.

I also tried to move up to 3.4.1 and found issues. I have the threaded ncurses, so the module is called ncursest. My hack way to make this work was to link ncursest to ncurses and the libs as well with sym links...but it was still missing -ltinfot and broke because it could not find "cbreak" object. The better way would be to teach cmake how to find ncursest.

Here is what I used to build ncurses:
bash ./configure --build=${GNU_MACH_BUILD} \
            --prefix=${INSTARGET} \
            --bindir=${BINTARGET} \
            --libdir=${LIBTARGET} \
            --libexecdir=${LIBEXTARGET} \
            --includedir=${INCTARGET} \
            --with-libtool \
            --with-pkg-config \
            --with-shared \
            --with-cxx-shared \
            --with-termlib \
            --with-pthread \
            --enable-rpath \
            --enable-ext-colors


One last thing. I had -I${INCTARGET} -I${INC64TARGET} which SHOULD find stuff first in ${INCTARGET} FIRST, but for some reason the CURL includes came from ${INC64TARGET} instead...maybe because it was checked 2nd and the last one won the race? This was a big deal because:

$ diff local-201601/include/curl/curlbuild.h local64-201601/include/curl/curlbuild.h
136c136
< #define CURL_PULL_STDINT_H 1
---
> /* #undef CURL_PULL_STDINT_H */
143c143
< #define CURL_PULL_INTTYPES_H 1
---
> /* #undef CURL_PULL_INTTYPES_H */
163c163
< #define CURL_SIZEOF_LONG 4
---
> #define CURL_SIZEOF_LONG 8
175c175
< #define CURL_TYPEOF_CURL_OFF_T int64_t
---
> #define CURL_TYPEOF_CURL_OFF_T long
181c181
< #define CURL_FORMAT_CURL_OFF_T "lld"
---
> #define CURL_FORMAT_CURL_OFF_T "ld"
184c184
< #define CURL_FORMAT_CURL_OFF_TU "llu"
---
> #define CURL_FORMAT_CURL_OFF_TU "lu"
187c187
< #define CURL_FORMAT_OFF_T "%lld"
---
> #define CURL_FORMAT_OFF_T "%ld"
193c193
< #define CURL_SUFFIX_CURL_OFF_T LL
---
> #define CURL_SUFFIX_CURL_OFF_T L
196c196
< #define CURL_SUFFIX_CURL_OFF_TU ULL
---
> #define CURL_SUFFIX_CURL_OFF_TU UL
(0040233)
R Jones   
2016-01-14 09:49   
I moved up to 3.4.1 and got it compiled...mostly due to the test failures on 3.3.1...and I am running the "make test" on 3.4.1 now... hoping those failures are resolved....but:

115/410 Test 0000115: BuildDepends .....................................***Failed 25.17 sec
173/410 Test #173: CTestTestUpload ..................................***Failed Required regular expression not found.Regex=[Upload\.xml] 9.60 sec
174/410 Test #174: CTestCoverageCollectGCOV .........................***Failed Required regular expression not found.Regex=[PASSED with correct output.*Testing/CoverageInfo/main.cpp.gcov] 8.38 sec
193/410 Test 0000193: CMakeCommands.target_link_libraries ..............***Failed 18.86 sec


These tests failed in 3.3.1 also. What do you need from the logs to see what's happening?

It's still chugging on the tests. 205/410 done.
(0040235)
R Jones   
2016-01-14 10:33   
I opened bug 0015920 to cover the 3.4.1 test failures...
(0040236)
R Jones   
2016-01-14 10:34   
0015920
(0040239)
Brad King   
2016-01-14 11:01   
Re 0015916:0040232:

> Are you saying that if I use "--prefix=${INSTARGET}" that ${INSTARGET}/include will be checked before /usr/include or /usr/local/include?

Not quite. In CMake build systems we prefer to have CMake add all -I flags, so to change the search order one should instead convince CMake to find the preferred include directories. The list of prefixes searched is built here:

 https://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/Platform/UnixPaths.cmake;hb=v3.4.1#l32 [^]

and the install prefix is included, but not before system prefixes. CMake projects usually (indirectly) use the find_path() command to search for headers:

 https://cmake.org/cmake/help/v3.4/command/find_path.html [^]

That documentation lists the search order. You could add CMAKE_INSTALL_PREFIX=${INSTARGET} to your environment to get it searched first.

If needed we could discuss this further on the mailing list as I requested in 0015920:0040238.
(0041290)
Kitware Robot   
2016-06-10 14:21   
This issue tracker is no longer used. Further discussion of this issue may take place in the current CMake Issues page linked in the banner at the top of this page.