[CMake] how to force assign sequence in multi-thread in cmake
Yu Jing
yujing5b5d at gmail.com
Tue Sep 16 07:58:44 EDT 2014
I am trying … That’s a very clear organization
On Sep 16, 2014, at 19:57, Micha Hergarden <micha.hergarden at gmail.com> wrote:
> Hello Yu,
>
> That is because of the following lines:
>
> AUX_SOURCE_DIRECTORY(. DIR_TARGET)
> ADD_LIBRARY (relfiles ${DIR_TARGET})
>
> I think you may be misusing the aux_source_directory command. I have tested your setup on a linux machine with -j8 and it still builds. Did you take a look at the openchemistry setup?
>
> Regards,
> Micha
>
> On 09/16/2014 01:42 PM, Yu Jing wrote:
>> Hello Micha,
>> It seems still not work….
>>
>> On Sep 16, 2014, at 17:36, Micha Hergarden <micha.hergarden at gmail.com> wrote:
>>
>>> Hello Yu,
>>>
>>> It seems I too needed another trick to make it work:
>>> set_property(TARGET CRFPP_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)
>>> set_property(TARGET LEVELDB_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)
>>>
>>> ADD_DEPENDENCIES(cmake_sample iniparser_static CRFPP_EX_PROJ LEVELDB_EX_PROJ)
>>>
>>> By default the ExternalProject_Add will add that target to the ALL target, and that confuses the Make dependency checker. I have removed those dependencies in my project. On the whole I agree with Marcel Loose and Petr Kmoch however. Combining their solution makes for a 'cleaner' project. To be honest, I am halfway of migrating my project to a superbuild as well. Take a look at the openchemistry project if you would like to see an example of that: https://github.com/OpenChemistry/openchemistry
>>>
>>> Regards,
>>> Micha Hergarden
>>>
>>>
>>> On 09/16/2014 11:04 AM, Marcel Loose wrote:
>>>> Hi Yu,
>>>>
>>>> I think you need to add an explicit dependency of main.cc on the "generated" (well, not really generated, but installed) header file crfpp.h. CMake has no clue as to what files are being compiled/installed by your external project, so you have to make this explicit.
>>>>
>>>> HTH,
>>>> Marcel Loose.
>>>>
>>>>
>>>> On 16/09/14 10:41, Yu Jing wrote:
>>>>> Hello Micha ,
>>>>>
>>>>> It seems doesn’t work.
>>>>> I updated code , and move the external project to src , and it still not work.
>>>>>
>>>>> what should be noticed is :
>>>>> 1. If I just use
>>>>> $ make
>>>>> as make command ,all styles are work
>>>>> 2. If I use in multi jobs
>>>>> $ make -j8
>>>>> It will run abnormal (not desired sequence) .
>>>>>
>>>>>
>>>>> On Sep 16, 2014, at 15:27, Micha Hergarden <micha.hergarden at gmail.com> wrote:
>>>>>
>>>>>> It may be that you have line 63 and 64 the wrong way around:
>>>>>> ADD_SUBDIRECTORY(src)
>>>>>> ADD_SUBDIRECTORY(lib)
>>>>>>
>>>>>> The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
>>>>>> Does reversing the directories help?
>>>>>>
>>>>>> Regards,
>>>>>> Micha
>>>>>>
>>>>>> On 09/16/2014 09:17 AM, Yu Jing wrote:
>>>>>>> I am in OSX 10.9.4 , a sample in github is : https://github.com/yujing5b5d/cmake_sample
>>>>>>> after git clone this project , a operation like this :
>>>>>>> ------------------------------------------------------------------------------------------------------------
>>>>>>>
>>>>>>> yu:cmake_sample yu$ mkdir build
>>>>>>> yu:cmake_sample yu$ cd build/
>>>>>>> yu:build yu$ cmake ..
>>>>>>> …. # skip some useless output
>>>>>>> -- Configuring done
>>>>>>> -- Generating done
>>>>>>> -- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
>>>>>>> yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
>>>>>>> Scanning dependencies of target LEVELDB_EX_PROJ
>>>>>>> Scanning dependencies of target iniparser
>>>>>>> Scanning dependencies of target relfiles
>>>>>>> Scanning dependencies of target CRFPP_EX_PROJ
>>>>>>> Scanning dependencies of target iniparser_static
>>>>>>> [ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
>>>>>>> Creating directories for 'CRFPP_EX_PROJ'
>>>>>>> Creating directories for 'LEVELDB_EX_PROJ'
>>>>>>> Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
>>>>>>> Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
>>>>>>> [ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
>>>>>>> Performing download step (git clone) for 'CRFPP_EX_PROJ'
>>>>>>> /Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
>>>>>>> #include "crfpp.h" // crfpp
>>>>>>> ^
>>>>>>> Cloning into 'CRFPP_EX_PROJ'...
>>>>>>> Cloning into 'LEVELDB_EX_PROJ'...
>>>>>>> Linking C static library ../libiniparser.a
>>>>>>> Linking C shared library ../libiniparser.dylib
>>>>>>> [ 35%] [ 35%] Built target iniparser_static
>>>>>>> Built target iniparser
>>>>>>> Scanning dependencies of target cmake_sample
>>>>>>> [ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
>>>>>>> /Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
>>>>>>> #include "crfpp.h" // crfpp
>>>>>>> ^
>>>>>>> 1 error generated.
>>>>>>> 1 error generated.
>>>>>>> make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
>>>>>>> make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
>>>>>>> make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
>>>>>>> make[1]: *** Waiting for unfinished jobs....
>>>>>>> make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
>>>>>>> ….
>>>>>>> ------------------------------------------------------------------------------------------------------------
>>>>>>>
>>>>>>> BE CAREFUL OF THIS LINE :
>>>>>>> >> yu:build yu$ make -j8
>>>>>>>
>>>>>>> the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h header to a special path, after this process , we can use #include “crfpp.h" ,and If I use
>>>>>>> make -j8
>>>>>>> this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.
>>>>>>>
>>>>>>> Of course , I’m not sure is this my misuse this project .
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sep 16, 2014, at 14:55, Micha Hergarden <micha.hergarden at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello all,
>>>>>>>>
>>>>>>>> I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.
>>>>>>>>
>>>>>>>> What exactly does not work? Is the external project not build at all, or just not in time?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Micha
>>>>>>>>
>>>>>>>> On 09/16/2014 08:30 AM, Petr Kmoch wrote:
>>>>>>>>> Hi.
>>>>>>>>>
>>>>>>>>> I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.
>>>>>>>>>
>>>>>>>>> Petr
>>>>>>>>>
>>>>>>>>> On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <yujing5b5d at gmail.com> wrote:
>>>>>>>>> I had asked in http://stackoverflow.com/questions/25841602 , someone told me maybe here is a better place to ask.
>>>>>>>>>
>>>>>>>>> I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .
>>>>>>>>>
>>>>>>>>> firstly , I add a extenal project like this:
>>>>>>>>>
>>>>>>>>> EXTERNALPROJECT_ADD(
>>>>>>>>> CRFPP_EX_PROJ
>>>>>>>>> GIT_REPOSITORY git at github.com:yujing5b5d/crfpp.git
>>>>>>>>> PREFIX ${CMAKE_CURRENT_BINARY_DIR}
>>>>>>>>> CONFIGURE_COMMAND ./configure
>>>>>>>>> BUILD_COMMAND make -j8
>>>>>>>>> BUILD_IN_SOURCE 1
>>>>>>>>> INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include
>>>>>>>>> )
>>>>>>>>> this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.
>>>>>>>>>
>>>>>>>>> and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .
>>>>>>>>>
>>>>>>>>> INCLUDE_DIRECTORIES(
>>>>>>>>> ${PROJECT_SOURCE_DIR}/include
>>>>>>>>> ${PROJECT_BINARY_DIR}/include
>>>>>>>>> )
>>>>>>>>> finally , when I compile the main project , code like this :
>>>>>>>>>
>>>>>>>>> ADD_EXECUTABLE(cmake_sample main.cc)
>>>>>>>>> ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
>>>>>>>>> TARGET_LINK_LIBRARIES(cmake_sample crfpp)
>>>>>>>>> In general , if I just build a build folder, compile like this :
>>>>>>>>>
>>>>>>>>> cmake .
>>>>>>>>> .
>>>>>>>>> <
>>>>>>>>> /span>
>>>>>>>>> make
>>>>>>>>> It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :
>>>>>>>>>
>>>>>>>>> cmake
>>>>>>>>> .
>>>>>>>>> .
>>>>>>>>> make -j8
>>>>>>>>> It will not works because my main.cc and the external project are processed at same time, so it report a error like this :
>>>>>>>>>
>>>>>>>>> /Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
>>>>>>>>> #include "crfpp.h" // crfpp
>>>>>>>>> ^
>>>>>>>>> 1 error generated.
>>>>>>>>> This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .
>>>>>>>>>
>>>>>>>>> My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote
>>>>>>>>>
>>>>>>>>> ADD_DEPENDENCIES(
>>>>>>>>> <
>>>>>>>>> span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)
>>>>>>>>> but it seems not works ?
>>>>>>>>>
>>>>>>>>> Thanks for any help.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Powered by www.kitware.com
>>>>>>>>>
>>>>>>>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>>>>>>>
>>>>>>>>> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>>>>>>>>>
>>>>>>>>> CMake Support: http://cmake.org/cmake/help/support.html
>>>>>>>>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>>>>>>>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>>>>>>>>
>>>>>>>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>>>>>>>
>>>>>>>>> Follow this link to subscribe/unsubscribe:
>>>>>>>>> http://public.kitware.com/mailman/listinfo/cmake
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>> Powered by www.kitware.com
>>>
>>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>>
>>> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>>>
>>> CMake Support: http://cmake.org/cmake/help/support.html
>>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>>
>>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/cmake
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/cmake/attachments/20140916/450ed512/attachment-0001.html>
More information about the CMake
mailing list