From jerry.c.t at web.de Mon Aug 1 03:58:27 2016 From: jerry.c.t at web.de (jerry.c.t at web.de) Date: Mon, 1 Aug 2016 09:58:27 +0200 Subject: [cmake-developers] [PATCH] Fix FindOpenCL on Mac OS Message-ID: Hi, I tried the FindOpenCL.cmake with the imported target OpenCL::OpenCL and found out that it does not work on MacOS. The first problem is that it does not detect the version. In line 56 the path needs to be changed from ${OpenCL_INCLUDE_DIR}/OpenCL/cl.h" to ?${OpenCL_INCLUDE_DIR}/Headers/cl.h? because OpenCL_INCLUDE_DIR points to the root of the OpenCL framework directory and the Headers are located under Headers. Also there seems to be a bug in find_package_handle_standard_args (?). Because without the change above - when there is no version found - the variable OpenCL_VERSION_STRING is empty. But an empty version string does not let cmake fail. This is the output -- Looking for CL_VERSION_2_0 -- Looking for CL_VERSION_2_0 - not found -- Looking for CL_VERSION_1_2 -- Looking for CL_VERSION_1_2 - not found -- Looking for CL_VERSION_1_1 -- Looking for CL_VERSION_1_1 - not found -- Looking for CL_VERSION_1_0 -- Looking for CL_VERSION_1_0 - not found -- Found OpenCL: /System/Library/Frameworks/OpenCL.framework (Required is at least version "1.2") The other problems are with the imported locations in line 147ff. With the current solution the linker step fails because OpenCL_LIBRARY points to the root of the framework directory - i.e. /System/Library/Frameworks/OpenCL.framework. Long time ago there was a bug report [1] but the proposed solution was to have an if/else statement to handle the special case of Apple frameworks. So I based my solution on [2] and [3]. This works fine with Makefile and Ninja generators but the Xcode generator still fails. The problem now is that with Xcode 7 Apple switched the way how framework libraries work [5]. The final solution is based on [4] where the Apple framework case is changed to an INTERFACE IMPORTED library and the framework is stored in INTERFACE_LINK_LIBRARIES. This way cmake resolves the library path to -framework OpenCL. So with this patch I provide my final solution. Is it correct (it works at least for me now on Win, Linux and Apple)? What?s with this empty version bug? I don?t found a solution for this. jerry [1] https://cmake.org/Bug/view.php?id=14105 [2] https://github.com/Kitware/CMake/blob/master/Modules/FindGLUT.cmake [3] https://github.com/rpavlik/cmake-modules/blob/master/FindSDL2.cmake [4] http://public.kitware.com/pipermail/cmake/2016-April/063179.html [5] http://public.kitware.com/pipermail/cmake-developers/2015-August/026110.html -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-FindOpenCL-on-Mac-OS.patch Type: text/x-patch Size: 1505 bytes Desc: not available URL: From sergey.nikulov at gmail.com Mon Aug 1 05:22:26 2016 From: sergey.nikulov at gmail.com (Sergei Nikulov) Date: Mon, 1 Aug 2016 12:22:26 +0300 Subject: [cmake-developers] Strange behaviour with check_library_exists on Windows In-Reply-To: References: <20160728120304.GA17760@megas.kitware.com> Message-ID: 2016-07-28 17:10 GMT+03:00 Sergei Nikulov : > 2016-07-28 16:58 GMT+03:00 Sergei Nikulov : >> 2016-07-28 15:03 GMT+03:00 Ben Boeckel : >>> On Thu, Jul 28, 2016 at 13:33:35 +0300, Sergei Nikulov wrote: >>>> -- Looking for CertFreeCertificateContext in crypt32; >>>> -- Looking for CertFreeCertificateContext in crypt32; - found >>>> crypt32 found >>> >>> These are tested using try_compile. Is there anything interesting under >>> CMakeFiles/CMakeTmp/* when you configure with --debug-trycompile? It >>> should contain the example files used and output from the compiler. >>> >> >> Ben, >> >> So looks like name mangling differs for different targets. > > Yes. > Here the explanation > http://stackoverflow.com/questions/28062446/x64-dll-export-function-names > Hello Ben, Should I submit issue regarding this findings? Thank you. -- Best Regards, Sergei Nikulov From cmake at anteru.net Mon Aug 1 08:32:48 2016 From: cmake at anteru.net (=?UTF-8?Q?Matth=c3=a4us_G._Chajdas?=) Date: Mon, 1 Aug 2016 14:32:48 +0200 Subject: [cmake-developers] [PATCH] Fix FindOpenCL on Mac OS In-Reply-To: References: Message-ID: <2169c0f4-9c36-9091-b99b-cef60639d4ac@anteru.net> Hi Jerry, thanks for giving it a spin :) I don't have a Mac to test myself - as your changes are confined to macOS, they look safe to me. The version is not listed as a required variable, so that's why it passes through. If you pass in a version into your find_package call, it should bail out if no version is specified. I only got one minor nit-pick: Why did you change the line INTERFACE_INCLUDE_DIRECTORIES "${OpenCL_INCLUDE_DIR}") to use _DIR instead of _DIRS? I assume this was from testing, not because that changed something on macOS? Other than that, the patch looks good to me. I'll apply it this week. Cheers, Matth?us Am 01.08.2016 um 09:58 schrieb jerry.c.t at web.de: > Hi, > > I tried the FindOpenCL.cmake with the imported target OpenCL::OpenCL and found out that it does not work on MacOS. > > The first problem is that it does not detect the version. In line 56 the path needs to be changed from ${OpenCL_INCLUDE_DIR}/OpenCL/cl.h" to ?${OpenCL_INCLUDE_DIR}/Headers/cl.h? because OpenCL_INCLUDE_DIR points to the root of the OpenCL framework directory and the Headers are located under Headers. > > Also there seems to be a bug in find_package_handle_standard_args (?). Because without the change above - when there is no version found - the variable OpenCL_VERSION_STRING is empty. But an empty version string does not let cmake fail. This is the output > > -- Looking for CL_VERSION_2_0 > -- Looking for CL_VERSION_2_0 - not found > -- Looking for CL_VERSION_1_2 > -- Looking for CL_VERSION_1_2 - not found > -- Looking for CL_VERSION_1_1 > -- Looking for CL_VERSION_1_1 - not found > -- Looking for CL_VERSION_1_0 > -- Looking for CL_VERSION_1_0 - not found > -- Found OpenCL: /System/Library/Frameworks/OpenCL.framework (Required is at least version "1.2") > > The other problems are with the imported locations in line 147ff. With the current solution the linker step fails because OpenCL_LIBRARY points to the root of the framework directory - i.e. /System/Library/Frameworks/OpenCL.framework. Long time ago there was a bug report [1] but the proposed solution was to have an if/else statement to handle the special case of Apple frameworks. So I based my solution on [2] and [3]. This works fine with Makefile and Ninja generators but the Xcode generator still fails. The problem now is that with Xcode 7 Apple switched the way how framework libraries work [5]. The final solution is based on [4] where the Apple framework case is changed to an INTERFACE IMPORTED library and the framework is stored in INTERFACE_LINK_LIBRARIES. This way cmake resolves the library path to -framework OpenCL. > > So with this patch I provide my final solution. Is it correct (it works at least for me now on Win, Linux and Apple)? > > What?s with this empty version bug? I don?t found a solution for this. > > jerry > > [1] https://cmake.org/Bug/view.php?id=14105 > [2] https://github.com/Kitware/CMake/blob/master/Modules/FindGLUT.cmake > [3] https://github.com/rpavlik/cmake-modules/blob/master/FindSDL2.cmake > [4] http://public.kitware.com/pipermail/cmake/2016-April/063179.html > [5] http://public.kitware.com/pipermail/cmake-developers/2015-August/026110.html > > > From jerry.c.t at web.de Mon Aug 1 15:04:44 2016 From: jerry.c.t at web.de (jerry.c.t at web.de) Date: Mon, 1 Aug 2016 21:04:44 +0200 Subject: [cmake-developers] [PATCH] Fix FindOpenCL on Mac OS Message-ID: Hi, > The version is not listed as a required variable, so that's why it > passes through. If you pass in a version into your find_package call, it > should bail out if no version is specified. Sadly not. The current version in master does not fail when invoked with "find_package(OpenCL 1.2 REQUIRED)". My first email shows exactly the output. You see that it says "Required is at least version "1.2"" while also saying "Found OpenCL:..." while also no version was found. The patch now fails if invoked for example with find_package(OpenCL 2.0 REQUIRED). It seems that find_package_handle_standard does not work as expected when OpenCL_VERSION_STRING is empty. > I assume this was from testing, not > because that changed something on macOS? You are right _DIR and _DIRS are working. I think I only changed it because for example FindGLUT uses it, for the library the singular variant LIBRARY is used and I don't understand the difference of _DIR and _DIRS :) Jerry On 01.08.2016 14:32, Matth?us G. Chajdas wrote: > Hi Jerry, > > thanks for giving it a spin :) > > I don't have a Mac to test myself - as your changes are confined to > macOS, they look safe to me. > > The version is not listed as a required variable, so that's why it > passes through. If you pass in a version into your find_package call, it > should bail out if no version is specified. > > I only got one minor nit-pick: Why did you change the line > INTERFACE_INCLUDE_DIRECTORIES "${OpenCL_INCLUDE_DIR}") > to use _DIR instead of _DIRS? I assume this was from testing, not > because that changed something on macOS? > > Other than that, the patch looks good to me. I'll apply it this week. > > Cheers, > Matth?us > From masoudafrakhteh at etri.re.kr Mon Aug 1 20:24:43 2016 From: masoudafrakhteh at etri.re.kr (=?ks_c_5601-1987?B?uLa89rXl?=) Date: Tue, 2 Aug 2016 00:24:43 +0000 Subject: [cmake-developers] Problem with openCV with CUDA installation using CMAKE on PASCAL Architecture Message-ID: Hello First of all I apologize for the quality of my English as I am not a native speaker. It is not actually my first time facing this problem and I am really tired of searching for it on blogs this time. I have had a graphic card with MAXWEll architecture (listed in CUDA enabled cards on INVidia list) and could not generate the files without errors using CMAKE. I gave up then and used an older card with KEPLER architecture. That time, opencv with CUDA was successfully generated by CMAKE and was easily compiled with Visual C++ 2012 . I had no problem using OPENCV libraries for GPU. Now, I am back to the same problem for some reasons and have to use a newer card (GEFORCE GTX 1070, CUDA enabled) which is a graphic card with PASCAL architecture and I cannot find it where CUDA generation options are listed in CMAKE 3.6.1. Once, I did not change any parameters including CUDA_ARCH_BIN 1.0 2.0 , so on and could successfully generate without errors selecting KEPLER option, it is a PASCAL architecture though. I could even successfully re-build opencv files from generated files by CMAKE. However GPU functions were not working and I faced this error : ?OpenCV Error : Gpu API call (Invalid device symbol) in unknown function?. I searched a lot online and felt that PASCAL Architecture and KEPLER are not the same that such an error shows up. FYI, I have successfully done devicequery function on my PC now which confirms the successful installation of CUDA toolkit and required files here. In another effort, I tried to manually change the default values listed for CUDA_ARCH_BIN and wrote only 6.0 6.1 6.2 6.3 and repeated them for CUDA_ARCH_PTX. I left CUDA generation empty and received many warnings ,not errors. I tried to rebuild those CMAKE generated files in Visual C and got many failures consequently. Kindly let me know what to do so that I can get the right cmake files generated for PASCAL architecture only. Looking forward to hearing from you, Masoud -------------- next part -------------- An HTML attachment was scrubbed... URL: From nilsgladitz at gmail.com Tue Aug 2 04:47:23 2016 From: nilsgladitz at gmail.com (Nils Gladitz) Date: Tue, 2 Aug 2016 10:47:23 +0200 Subject: [cmake-developers] [Patch 3/5] Improved WIX support In-Reply-To: References: Message-ID: <78783b5d-1814-777b-bcf8-ef95857ce9e7@gmail.com> On 07/20/2016 03:58 PM, Stuermer, Michael SP/HZA-ZSEP wrote: > Using the patchfile support I managed to implement the service installation issue I had, so the unnecessary features from the last patch are removed now. > > I tested all patches separately and hope they work now. > > looking forward for feedback, > > best regrads, > Michael Patch 3 looks OK but I think I would prefer using WiX over CPack nomenclature in the variables you introduce. e.g. CPACK_WIX_ROOT_FEATURE_TITLE and CPACK_WIX_ROOT_FEATURE_DESCRIPTION over CPACK_WIX_ROOT_COMPONENT_DISPLAY_NAME and CPACK_WIX_ROOT_COMPONENT_DESCRIPTION given that CPack has no concept of a root component. Would you agree? Nils From michael.stuermer at schaeffler.com Tue Aug 2 07:29:33 2016 From: michael.stuermer at schaeffler.com (Stuermer, Michael SP/HZA-ZSEP) Date: Tue, 2 Aug 2016 11:29:33 +0000 Subject: [cmake-developers] [Patch 3/5] Improved WIX support In-Reply-To: <78783b5d-1814-777b-bcf8-ef95857ce9e7@gmail.com> References: <78783b5d-1814-777b-bcf8-ef95857ce9e7@gmail.com> Message-ID: <7f5a080d05af4dc996364e68bd5d7475@DE013666.schaeffler.com> > -----Original Message----- > From: Nils Gladitz [mailto:nilsgladitz at gmail.com] > Sent: Tuesday, August 02, 2016 10:47 AM > To: Stuermer, Michael SP/HZA-ZSEP; CMake Developers > Subject: Re: [cmake-developers] [Patch 3/5] Improved WIX support > > On 07/20/2016 03:58 PM, Stuermer, Michael SP/HZA-ZSEP wrote: > > > Using the patchfile support I managed to implement the service installation > issue I had, so the unnecessary features from the last patch are removed > now. > > > > I tested all patches separately and hope they work now. > > > > looking forward for feedback, > > > > best regrads, > > Michael > > Patch 3 looks OK but I think I would prefer using WiX over CPack > nomenclature in the variables you introduce. > > e.g. > > CPACK_WIX_ROOT_FEATURE_TITLE and > CPACK_WIX_ROOT_FEATURE_DESCRIPTION > > over > > CPACK_WIX_ROOT_COMPONENT_DISPLAY_NAME and > CPACK_WIX_ROOT_COMPONENT_DESCRIPTION > > given that CPack has no concept of a root component. > > Would you agree? > totally. > Nils Best Regards Michael St?rmer From nilsgladitz at gmail.com Tue Aug 2 08:17:02 2016 From: nilsgladitz at gmail.com (Nils Gladitz) Date: Tue, 2 Aug 2016 14:17:02 +0200 Subject: [cmake-developers] [Patch 3/5] Improved WIX support In-Reply-To: <7f5a080d05af4dc996364e68bd5d7475@DE013666.schaeffler.com> References: <78783b5d-1814-777b-bcf8-ef95857ce9e7@gmail.com> <7f5a080d05af4dc996364e68bd5d7475@DE013666.schaeffler.com> Message-ID: <2477b43e-f76c-354a-51ae-dba18fae085c@gmail.com> On 08/02/2016 01:29 PM, Stuermer, Michael SP/HZA-ZSEP wrote: >> -----Original Message----- >> From: Nils Gladitz [mailto:nilsgladitz at gmail.com] >> Sent: Tuesday, August 02, 2016 10:47 AM >> To: Stuermer, Michael SP/HZA-ZSEP; CMake Developers >> Subject: Re: [cmake-developers] [Patch 3/5] Improved WIX support >> Patch 3 looks OK but I think I would prefer using WiX over CPack >> nomenclature in the variables you introduce. >> >> e.g. >> >> CPACK_WIX_ROOT_FEATURE_TITLE and >> CPACK_WIX_ROOT_FEATURE_DESCRIPTION >> >> over >> >> CPACK_WIX_ROOT_COMPONENT_DISPLAY_NAME and >> CPACK_WIX_ROOT_COMPONENT_DESCRIPTION >> >> given that CPack has no concept of a root component. >> >> Would you agree? >> > totally. > Thanks. I added a release note, modified the commit message and merged to "next" for testing: https://cmake.org/gitweb?p=cmake.git;a=commit;h=e29bfbf2 Nils From brad.king at kitware.com Tue Aug 2 13:06:57 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:06:57 -0400 Subject: [cmake-developers] "Run ranlib on archives only if the tool is available" not in 3.6.1 release In-Reply-To: References: Message-ID: <45441dfd-9bae-3a16-8efc-fd4fb003177b@kitware.com> On 07/22/2016 04:24 PM, Ryan D Retting wrote: > I've been waiting for the following commit to get included in an official > release. I just checked today's 3.6.1 and it's still not there. 3.6.1 is a patch release. Only documentation updates and fixes for regressions from 3.5 are eligible for patch releases. The fix you linked was made in post-3.6 development and will be included in 3.7. Meanwhile the nightly binaries or a build from source can be used. -Brad From brad.king at kitware.com Tue Aug 2 13:07:17 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:07:17 -0400 Subject: [cmake-developers] Spurious warning with FindCUDA.cmake on Windows In-Reply-To: References: Message-ID: <7d221de6-181d-7e84-1fd5-f7f1d18fe167@kitware.com> On 07/22/2016 06:08 PM, Sorley, Stephen L. wrote: > I?ve attached a patch (based on master) that fixes the problem. Thanks, applied: FindCUDA: Do not look for librt on Windows https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=f4e979b1 -Brad From brad.king at kitware.com Tue Aug 2 13:07:46 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:07:46 -0400 Subject: [cmake-developers] [PATCH v3 6/7] Use Windows version of Directory::Load for MinGW too In-Reply-To: <8222e03b-4882-9a60-37f0-a45b8f233ca1@kitware.com> References: <20160706191213.16242-1-davispuh@gmail.com> <20160706191213.16242-6-davispuh@gmail.com> <8222e03b-4882-9a60-37f0-a45b8f233ca1@kitware.com> Message-ID: <0523a38f-1ebc-a4c7-0f84-24ae46330945@kitware.com> On 07/20/2016 10:27 AM, Brad King wrote: > On 07/06/2016 03:12 PM, D?vis Mos?ns wrote: >> Otherwise it would use POSIX functions which works only for ASCII paths. > > Thanks. Applied to KWSys first with minor tweaks: > > http://review.source.kitware.com/21349 After being rewritten by another contributor there the change is now in KWSys master. I've updated CMake's copy to get it: KWSys 2016-08-01 (560bcdbb) https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=6bc3073e -Brad From brad.king at kitware.com Tue Aug 2 13:08:23 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:08:23 -0400 Subject: [cmake-developers] patches from pkgsrc In-Reply-To: <20160728162612.9935-1-prlw1@cam.ac.uk> References: <20160728162612.9935-1-prlw1@cam.ac.uk> Message-ID: On 07/28/2016 12:26 PM, Patrick Welche wrote: > While looking at updating cmake at: > > http://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc/devel/cmake/README.html > > I see that there are some patches which look useful, which I attach in > this series. Thanks. I've applied the CMake parts here: cmELF: Port to SCO OpenServer 5.0.7/3.2 https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=16a3a735 SCO_SV: Enable so filename versioning https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=56539d89 I've applied the KWSys parts to its upstream: * http://review.source.kitware.com/21403 SystemInformation: Treat BSDs more uniformly * http://review.source.kitware.com/21404 SystemInformation: Treat Solaris the same as Linux * http://review.source.kitware.com/21405 SystemTools: Fix FileExists for some SCO OpenServer file permissions Once integrated there I can update CMake's copy. I left out the "Use correct cmake macro." change because it is wrong. The SIZEOF_VOID_P macro is defined by code in CMakeLists.txt using the value of CMAKE_SIZEOF_VOID_P. The former is a C++ preprocessor macro. The latter is only visible in CMake code. -Brad From brad.king at kitware.com Tue Aug 2 13:08:47 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:08:47 -0400 Subject: [cmake-developers] [PATCH] Let CMake know that clang uses the GCC toolchain. In-Reply-To: <1469828685-28388-1-git-send-email-aoe@google.com> References: <1469828685-28388-1-git-send-email-aoe@google.com> Message-ID: <65b663cd-4200-6140-651d-c630c89eb908@kitware.com> On 07/29/2016 05:44 PM, Chaoren Lin via cmake-developers wrote: > Clang on Windows uses the GCC toolchain, so we need to avoid using > backslashes for paths in response files. [snip] > strcmp(mf->GetSafeDefinition("CMAKE_C_COMPILER_ID"), "GNU") == 0 || > strcmp(mf->GetSafeDefinition("CMAKE_CXX_COMPILER_ID"), "GNU") == 0 || > strcmp(mf->GetSafeDefinition("CMAKE_C_SIMULATE_ID"), "GNU") == 0 || > - strcmp(mf->GetSafeDefinition("CMAKE_CXX_SIMULATE_ID"), "GNU") == 0) { Clang on Windows also comes in a cl-like variant whose response files may need backslashes. In the gcc-like variant the CMAKE_{C,CXX}_SIMULATE_ID should be detected as GNU and cause the existing code to work. What goes wrong in your use case? Thanks, -Brad From brad.king at kitware.com Tue Aug 2 13:08:59 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:08:59 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <1469828692-28544-1-git-send-email-aoe@google.com> References: <1469828692-28544-1-git-send-email-aoe@google.com> Message-ID: <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> On 07/29/2016 05:44 PM, Chaoren Lin via cmake-developers wrote: > From: Chaoren Lin > > Relative paths are difficult for an IDE to parse the output of > a build error. Thanks, applied: Ninja: Use full path for all source files https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d867f8a0 This is more consistent with the Makefile generator behavior too. -Brad From brad.king at kitware.com Tue Aug 2 13:09:16 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:09:16 -0400 Subject: [cmake-developers] [PATCH] Allow custom osx framework locations. In-Reply-To: <20160730144853.26891-1-david.keller@litchis.fr> References: <20160730144853.26891-1-david.keller@litchis.fr> Message-ID: <52dadc24-462d-c13b-198c-2b1ac53992d0@kitware.com> On 07/30/2016 10:48 AM, David Keller wrote: > This allows non-standard locations to be searched, e.g. brew, port. Thanks, applied: CMakeFindFrameworks: Allow custom framework locations https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=9970cdcb FYI, this module really only exists for legacy reasons. The find_* commands know how to search frameworks already, and support some variables that can be set to add more search locations. -Brad From brad.king at kitware.com Tue Aug 2 13:09:29 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:09:29 -0400 Subject: [cmake-developers] FindProtobuf backward compatibility issue for PROTOBUF_IMPORT_DIRS In-Reply-To: References: Message-ID: On 07/29/2016 11:10 PM, Konstantin Sinitsyn wrote: > After recent changes in FindProtobuf.cmake protobuf_generate_cpp > doesn't take into account PROTOBUF_IMPORT_DIRS (in upper case) anymore. [snip] > Here's the patch to fix this: Thanks, applied: FindProtobuf: Restore support for PROTOBUF_IMPORT_DIRS https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=5790d9b6 I've queued this for merge to 'release' for 3.6.2. -Brad From brad.king at kitware.com Tue Aug 2 13:09:52 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:09:52 -0400 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> Message-ID: <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> On 07/31/2016 03:56 PM, Sebastian Holtermann wrote: >> I have updated the patch for CMake 3.6.0 to fix >> https://gitlab.kitware.com/cmake/cmake/issues/16209 > Here is an updated patch. Thanks. However, it is too late to restore this change for the 3.6 series. 3.6.1 reverted the change. We can re-introduce it for 3.7 so that it can go through another round of release candidates. Please rebase on `master` and include all needed changes as if this change were being made for the first time, including the restoration of the tests. Thanks, -Brad From brad.king at kitware.com Tue Aug 2 13:11:04 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:11:04 -0400 Subject: [cmake-developers] [PATCH v4 4/4] For Windows encode process output to internally used encoding In-Reply-To: References: <20160707215406.16180-1-davispuh@gmail.com> <20160707215406.16180-4-davispuh@gmail.com> <6487f0ec-3be6-8024-74a0-791952e46207@kitware.com> <50c7aa80-72ec-686f-932b-e51b87700b4b@kitware.com> Message-ID: On 07/21/2016 08:43 PM, D?vis Mos?ns wrote: > With MultiByteToWideChar such partial char would be replaced with ? (U+003F) > or ? (U+FFFD). [snip] > Also could check if last character is ? and try again with one byte less. This may be a good middle ground. The excess bytes would then be buffered for inclusion at the beginning of the next block conversion. How are we to know the encoding being produced by the child? > from WaitForData we're getting data and length, and I assume that data > might not be null-terminated but kwsysEncoding_mbstowcs expects source to be > null-terminated and doesn't accept length. Okay, thanks. Using MultiByteToWideChar for Windows-specific code is fine. If we ever need to offer a similar generalization then we can provide a kwsysEncoding_mbsnrtowcs later. BTW, with all your changes to KWSys it may be easier to iterate if you contribute to KWSys directly. Please see instructions here: http://public.kitware.com/Wiki/KWSys/Git/Develop Thanks, -Brad From brad.king at kitware.com Tue Aug 2 13:11:14 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:11:14 -0400 Subject: [cmake-developers] [PATCH v5] For consoles output on Windows use our own std::streambuf In-Reply-To: <20160727165405.22242-1-davispuh@gmail.com> References: <20160710021230.10618-1-davispuh@gmail.com> <20160727165405.22242-1-davispuh@gmail.com> Message-ID: <4d48fffa-1d1e-c942-1bdb-265f7cc1ab08@kitware.com> On 07/27/2016 12:54 PM, D?vis Mos?ns wrote: > Currently Microsoft's C++ libraries implementation of std::cout/cerr > can't output Unicode characters but only ASCII or ANSI if locale is set > so we implement and use our own ConsoleBuf which can output Unicode > characters to console and it doesn't matter what locale or console's > codepage is set. Thanks. I've added this for review in upstream KWSys: http://review.source.kitware.com/21407 Please see instructions at http://public.kitware.com/Wiki/KWSys/Git/Develop to sign up so we can iterate on the change there. Thanks, -Brad From brad.king at kitware.com Tue Aug 2 13:38:15 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 13:38:15 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> Message-ID: On 08/02/2016 01:08 PM, Brad King wrote: > Ninja: Use full path for all source files > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d867f8a0 I had to revert that because it causes the Qt*Autogen tests to fail. Also it may break builds on Windows. There are multiple problems: * ConvertToNinjaPath does more than just relative path conversions. It also converts slashes on Windows. Furthermore, it deals with CMAKE_NINJA_OUTPUT_PATH_PREFIX, though that won't matter for absolute paths. * Calls to AddAssumedSourceDependencies and HasCustomCommandOutput later in cmNinjaTargetGenerator::WriteObjectBuildStatement expect to be given paths that come out of ConvertToNinjaPath. * Ninja does not seem to be able to reconcile a dependency on an absolute path with a build statement providing that path by relative path. Custom commands that generate source files list their outputs with ConvertToNinjaPath. Basically everything assumes that all paths written as inputs or outputs of Ninja build statements have gone through the ConvertToNinjaPath method. The special case for "RC" in the existing code is likely buggy. Ninja always uses the location of the build.ninja file as the current working directory, so an IDE that sees a relative path in an error message can interpret it relative to that. -Brad From aoe at google.com Tue Aug 2 14:11:44 2016 From: aoe at google.com (Chaoren Lin) Date: Tue, 2 Aug 2016 11:11:44 -0700 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> Message-ID: > Ninja always uses the location of the build.ninja file as > the current working directory, so an IDE that sees a relative > path in an error message can interpret it relative to that. Our IDE has no knowledge of the build.gradle file let alone where it is located, it just sees the build output and tries to located the mentioned files. Similarly, the :make command in vim will have no knowledge of build.gradle, and only tries to parse the build errors directly. Would changing ConvertToNinjaPath to output absolute paths paths work? On Tue, Aug 2, 2016 at 10:38 AM, Brad King wrote: > On 08/02/2016 01:08 PM, Brad King wrote: > > Ninja: Use full path for all source files > > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d867f8a0 > > I had to revert that because it causes the Qt*Autogen tests to > fail. Also it may break builds on Windows. > > There are multiple problems: > > * ConvertToNinjaPath does more than just relative path conversions. > It also converts slashes on Windows. Furthermore, it deals with > CMAKE_NINJA_OUTPUT_PATH_PREFIX, though that won't matter for > absolute paths. > > * Calls to AddAssumedSourceDependencies and HasCustomCommandOutput > later in cmNinjaTargetGenerator::WriteObjectBuildStatement expect > to be given paths that come out of ConvertToNinjaPath. > > * Ninja does not seem to be able to reconcile a dependency on > an absolute path with a build statement providing that path > by relative path. Custom commands that generate source files > list their outputs with ConvertToNinjaPath. > > Basically everything assumes that all paths written as inputs > or outputs of Ninja build statements have gone through the > ConvertToNinjaPath method. The special case for "RC" in the > existing code is likely buggy. > > Ninja always uses the location of the build.ninja file as > the current working directory, so an IDE that sees a relative > path in an error message can interpret it relative to that. > > -Brad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Tue Aug 2 14:21:23 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 14:21:23 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> Message-ID: On 08/02/2016 02:11 PM, Chaoren Lin wrote: > Would changing ConvertToNinjaPath to output absolute paths paths work? No, that will make all paths absolute. Ninja's design is pretty clear in that it prefers canonical relative paths when possible. Note that the conversion to a relative path by ConvertToNinjaPath is not unconditional. In an out-of-source build the paths to the source files in the source tree will be absolute. --- Compilers know the absolute path to all the files in a translation unit since the preprocessor was able to read them. They could be taught to produce absolute paths in their error messages to work better for IDEs regardless of how the build system invokes them. I've never really understood why they don't make error messages and debug info hold absolute paths. For some reason relative paths in these places are always blamed on the build system when in fact the compiler could easily produce them. -Brad From konstantin at podsvirov.pro Tue Aug 2 14:17:11 2016 From: konstantin at podsvirov.pro (Konstantin Podsvirov) Date: Tue, 02 Aug 2016 21:17:11 +0300 Subject: [cmake-developers] [ISSUE] Bump to 3.2 and continue... In-Reply-To: <2255351469825804@web5g.yandex.ru> References: <2255351469825804@web5g.yandex.ru> Message-ID: <126961470161831@web22o.yandex.ru> 29.07.2016, 23:56, "Konstantin Podsvirov" : > Hello CMake developers, > > I found bug: > > https://gitlab.kitware.com/cmake/cmake/issues/16220 > > I found it, but I do not want to deal with it ... Okay, I have solution. May be attached patch will be more useful then MR (https://gitlab.kitware.com/cmake/cmake/merge_requests/51). > -- > Regards, > Konstantin Podsvirov > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers Regards, Konstantin Podsvirov -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Remove-using-of-continue-method.patch Type: text/x-diff Size: 1646 bytes Desc: not available URL: From brad.king at kitware.com Tue Aug 2 14:30:03 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 2 Aug 2016 14:30:03 -0400 Subject: [cmake-developers] cmake -E capabilities [attempt 2] In-Reply-To: References: <1469105598.765.32.camel@qt.io> Message-ID: On 07/29/2016 06:01 AM, Tobias Hunger wrote: > I pushed an update that takes most of the feedback into account. Thanks. I've applied the extra generator refactoring parts first: Refactor extra generator registration to use factories https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=bc44627b Report more information about extra generators in generator factories https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=034caa27 Once that tests cleanly I'll merge it to `master` and then we can rebase the remaining changes on it. Thanks, -Brad From ben.boeckel at kitware.com Tue Aug 2 14:33:02 2016 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Tue, 2 Aug 2016 14:33:02 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> Message-ID: <20160802183302.GA15133@megas.kitware.com> On Tue, Aug 02, 2016 at 11:11:44 -0700, Chaoren Lin via cmake-developers wrote: > Similarly, the :make command in vim will have no knowledge of build.gradle, > and only tries to parse the build errors directly. FWIW, I haven't encountered any problems with Vim's :make with CMake's generated Ninja files in out-of-source builds for in-source or generated files (I almost never do in-source builds). --Ben From davispuh at gmail.com Tue Aug 2 16:04:48 2016 From: davispuh at gmail.com (=?UTF-8?B?RMSBdmlzIE1vc8SBbnM=?=) Date: Tue, 2 Aug 2016 23:04:48 +0300 Subject: [cmake-developers] [PATCH v4 4/4] For Windows encode process output to internally used encoding In-Reply-To: References: <20160707215406.16180-1-davispuh@gmail.com> <20160707215406.16180-4-davispuh@gmail.com> <6487f0ec-3be6-8024-74a0-791952e46207@kitware.com> <50c7aa80-72ec-686f-932b-e51b87700b4b@kitware.com> Message-ID: 2016-08-02 20:11 GMT+03:00 Brad King : > > How are we to know the encoding being produced by the child? > There isn't any reliable way to detect it, we just have to know what particular application uses. Also there aren't any standard or API to determine it but generally most applications use console's code page. Of course not all, it could be OEM, ANSI or something else. If application uses console's code page for pipes then app.exe | echo will output correctly in cmd and piping in other application which also uses it will work. Here our issue is that when CMake checks for compiler path it gets output from MSVC and it will give this path in console's codepage which will be incorrectly interpreted as UTF-8 so we need to decode it first. If there will some other application which uses different encoding then it will need separate handling for that. From aoe at google.com Tue Aug 2 16:42:51 2016 From: aoe at google.com (Chaoren Lin) Date: Tue, 2 Aug 2016 13:42:51 -0700 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <20160802183302.GA15133@megas.kitware.com> References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> <20160802183302.GA15133@megas.kitware.com> Message-ID: The relative path is used whenever the build directory is under the source directory (e.g., under src/build/), so it could still be out of source. > FWIW, I haven't encountered any problems with Vim's :make with CMake's > generated Ninja files in out-of-source builds for in-source or generated > files (I almost never do in-source builds). Interesting. It appears that if you do :set makeprg=ninja\ -C\ /path/to/build | make then ninja will output this: > ninja: Entering directory `/path/to/build' and vim interprets that to be the working directory. However, if you use these: :set makeprg=cd\ /path/to/build\ &&\ ninja | make :set makeprg=cmake\ --build\ /path/to/build | make then the "Entering directory" line wouldn't appear, and vim won't know what to do. Our IDE uses "cmake --build /path/to/build" to invoke the build, so that we only have to deal with CMake and not the underlying build system. Compilers know the absolute path to all the files in a translation unit > since the preprocessor was able to read them. They could be taught to > produce absolute paths in their error messages to work better for IDEs > regardless of how the build system invokes them. I would love to see this too, but it's much easier to fix the problem in one place (CMake, or even just Ninja), than to fix that for every compiler. Maybe I can try to fix ninja to resolve paths before sending them to the compiler. Or maybe just add a flag to ConvertToNinjaPath or something. On Tue, Aug 2, 2016 at 11:33 AM, Ben Boeckel wrote: > On Tue, Aug 02, 2016 at 11:11:44 -0700, Chaoren Lin via cmake-developers > wrote: > > Similarly, the :make command in vim will have no knowledge of > build.gradle, > > and only tries to parse the build errors directly. > > FWIW, I haven't encountered any problems with Vim's :make with CMake's > generated Ninja files in out-of-source builds for in-source or generated > files (I almost never do in-source builds). > > --Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steveire at gmail.com Tue Aug 2 16:54:38 2016 From: steveire at gmail.com (Stephen Kelly) Date: Tue, 02 Aug 2016 22:54:38 +0200 Subject: [cmake-developers] cmake -E capabilities [attempt 2] References: <1469105598.765.32.camel@qt.io> Message-ID: <57a10892.c186c20a.af225.792c@mx.google.com> Tobias Hunger wrote: > Hi Stephen, > > thanks for taking the time to do such a thorough review! Actually my review wasn't thorough at all. I didn't try to review the branch further. The NewFactory methods in your patch don't return a new'd object, but instead return static locals. The regular generators NewFactory methods don't work that way, so you're introducing a pattern which is different to what already exists and the commit message doesn't say why. Maybe there's a reason, but I don't know what it is. In the future, no one else will know either. There are lots of things in there for which your intent is unclear. That's why splitting and writing descriptive commit messages is valuable. You might find https://vimeo.com/172882423 to be a good introduction to this. I have more to ask about your first commit (and why the second commit is split out). I stopped my review at recommending it be split to see what it is hiding. > Added const to some of them:-) Hope I caught all. cmake::ReportCapabilities() should be const, right? >> * and a whitespace change that should be squashed into the commit that >> introduces it > > I used Utilities/Scripts/clang-format.bash to do the formatting, so > that should not be an issue. I just reran that on all commits. Maybe I > forgot it in a commit or something before. Maybe. Running the script on all commits would be the fix anyway. >> * The CMAKE_BUILD_WITH_CMAKE macro should be in cmcmd.cxx wrapping the >> capabilities handling: >> >> #if defined(CMAKE_BUILD_WITH_CMAKE) >> else if (args[1] == "capabilities") { >> cmake cm; >> std::cout << cm.ReportCapabilities(); >> return 0; >> } >> #endif > > Why? Because it's a bit odd to return 0 if capabilities can't be reported. I missed that the contents of that method don't compile in bootstrap mode though without the define. >> As it is, it is doing many different things, none of which are mentioned >> in the commit message, and some of which it probably shouldn't be doing. >> >> For example renaming GetExtraGeneratorName to >> GetExternalMakefileProjectGeneratorName is probably not needed. If you >> really want to do it, then it should be in its own commit with its own >> commit message which justifies the change. As it is, it adds noise to the >> big commit and makes it harder to review. Minimal is always better with >> commits which do refactoring like that. > > I undid that change. That is one of the things that I originally > removed and then realized last minute that it is needed somehow. So I > added it, not realizing that I had removed similar functionality > earlier. This doesn't happen if you split commits. Splitting makes these things visible to you and you can decide whether they were intentional or not at that point. >> A general good rule of thumb (which helps reviews, and makes things >> bisectable in the future) is to do one thing per commit. > > I agree that this is the ideal we all should all strive for, but you > are not going to get that from me anytime soon. Calling it an 'ideal' isn't really the right mindset. It's easy, and it's generally done by all other cmake contributors. > At least not in the cmake codebase. I don't know what this part means. It seems somehow disdainful, but I might be missing something. Something to talk about in person perhaps. Thanks, Steve. From steveire at gmail.com Tue Aug 2 16:48:54 2016 From: steveire at gmail.com (Stephen Kelly) Date: Tue, 02 Aug 2016 22:48:54 +0200 Subject: [cmake-developers] cmake -E capabilities [attempt 2] References: <1469105598.765.32.camel@qt.io> Message-ID: Tobias Hunger wrote: > Hi Stephen, > > thanks for taking the time to do such a thorough review! Actually my review wasn't thorough at all. I didn't try to review it further. The NewFactory methods in your patch don't return a new'd object, but instead return static locals. The regular generators NewFactory methods don't work that way, so you're introducing a pattern which is different to what already exists and the commit message doesn't say why. Maybe there's a reason, but I don't know what it is. In the future, no one else will know either. There are lots of things in there for which your intent is unclear. That's why splitting and writing descriptive commit messages is valuable. You might find https://vimeo.com/172882423 to be a good introduction to this. I have more to ask about your first commit (and why the second commit is split out). I stopped my review at recommending it be split to see what it is hiding. > Added const to some of them:-) Hope I caught all. cmake::ReportCapabilities() should be const, right? >> * and a whitespace change that should be squashed into the commit that >> introduces it > > I used Utilities/Scripts/clang-format.bash to do the formatting, so > that should not be an issue. I just reran that on all commits. Maybe I > forgot it in a commit or something before. Maybe. Running the script on all commits would be the fix anyway. >> * The CMAKE_BUILD_WITH_CMAKE macro should be in cmcmd.cxx wrapping the >> capabilities handling: >> >> #if defined(CMAKE_BUILD_WITH_CMAKE) >> else if (args[1] == "capabilities") { >> cmake cm; >> std::cout << cm.ReportCapabilities(); >> return 0; >> } >> #endif > > Why? Because it's a bit odd to return 0 if capabilities can't be reported. I missed that the contents of that method don't compile in bootstrap mode though without the define. >> As it is, it is doing many different things, none of which are mentioned >> in the commit message, and some of which it probably shouldn't be doing. >> >> For example renaming GetExtraGeneratorName to >> GetExternalMakefileProjectGeneratorName is probably not needed. If you >> really want to do it, then it should be in its own commit with its own >> commit message which justifies the change. As it is, it adds noise to the >> big commit and makes it harder to review. Minimal is always better with >> commits which do refactoring like that. > > I undid that change. That is one of the things that I originally > removed and then realized last minute that it is needed somehow. So I > added it, not realizing that I had removed similar functionality > earlier. This doesn't happen if you split commits. Splitting makes these things visible to you and you can decide whether they were intentional or not at that point. >> A general good rule of thumb (which helps reviews, and makes things >> bisectable in the future) is to do one thing per commit. > > I agree that this is the ideal we all should all strive for, but you > are not going to get that from me anytime soon. Calling it an 'ideal' isn't really the right mindset. It's easy, and it's generally done by all other cmake contributors. > At least not in the cmake codebase. I don't know what this part means. It seems somehow disdainful, but I might be missing something. Something to talk about in person perhaps. Thanks, Steve. From aoe at google.com Tue Aug 2 18:59:10 2016 From: aoe at google.com (Chaoren Lin) Date: Tue, 2 Aug 2016 15:59:10 -0700 Subject: [cmake-developers] [PATCH] Let CMake know that clang uses the GCC toolchain. In-Reply-To: <65b663cd-4200-6140-651d-c630c89eb908@kitware.com> References: <1469828685-28388-1-git-send-email-aoe@google.com> <65b663cd-4200-6140-651d-c630c89eb908@kitware.com> Message-ID: > In the gcc-like variant the CMAKE_{C,CXX}_SIMULATE_ID > should be detected as GNU and cause the existing code to work. Huh, it doesn't look like that variable is set automatically when using clang with the gcc toolchain. Is that supposed to happen? It should be set manually (say, in a toolchain file)? On Tue, Aug 2, 2016 at 10:08 AM, Brad King wrote: > On 07/29/2016 05:44 PM, Chaoren Lin via cmake-developers wrote: > > Clang on Windows uses the GCC toolchain, so we need to avoid using > > backslashes for paths in response files. > [snip] > > strcmp(mf->GetSafeDefinition("CMAKE_C_COMPILER_ID"), "GNU") == 0 > || > > strcmp(mf->GetSafeDefinition("CMAKE_CXX_COMPILER_ID"), "GNU") == > 0 || > > strcmp(mf->GetSafeDefinition("CMAKE_C_SIMULATE_ID"), "GNU") == 0 > || > > - strcmp(mf->GetSafeDefinition("CMAKE_CXX_SIMULATE_ID"), "GNU") == > 0) { > > Clang on Windows also comes in a cl-like variant whose response files may > need backslashes. In the gcc-like variant the CMAKE_{C,CXX}_SIMULATE_ID > should be detected as GNU and cause the existing code to work. What goes > wrong in your use case? > > Thanks, > -Brad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Wed Aug 3 08:56:12 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 3 Aug 2016 08:56:12 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> <20160802183302.GA15133@megas.kitware.com> Message-ID: On 08/02/2016 04:42 PM, Chaoren Lin wrote: > Our IDE uses "cmake --build /path/to/build" to invoke the build, > so that we only have to deal with CMake and not the underlying build system. If you know /path/to/build why not just interpret relative paths with respect to it as a heuristic? If the paths exist then use them. > Maybe I can try to fix ninja to resolve paths before sending them to the compiler. The ninja build tool itself does not have enough information to make such a transformation. CMake does though. See below. > Or maybe just add a flag to ConvertToNinjaPath or something. ConvertToNinjaPath is called from several places that don't all know whether the file they are passing is a source file for compilation or not. Another approach is to use ConvertToNinjaPath for all the build.ninja inputs and outputs on build statements but use something other than $in for the source file on the compilation rule. Using an "IN_ABS = ..." variable could achieve this. -Brad From brad.king at kitware.com Wed Aug 3 08:56:38 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 3 Aug 2016 08:56:38 -0400 Subject: [cmake-developers] [PATCH] Let CMake know that clang uses the GCC toolchain. In-Reply-To: References: <1469828685-28388-1-git-send-email-aoe@google.com> <65b663cd-4200-6140-651d-c630c89eb908@kitware.com> Message-ID: <224843d8-c5b8-7d4d-6fca-727416eda120@kitware.com> On 08/02/2016 06:59 PM, Chaoren Lin wrote: >> In the gcc-like variant the CMAKE_{C,CXX}_SIMULATE_ID >> should be detected as GNU and cause the existing code to work. > > Huh, it doesn't look like that variable is set automatically when using > clang with the gcc toolchain. Is that supposed to happen? > It should be set manually (say, in a toolchain file)? It is supposed to be detected automatically. Whenever Clang is detected on Windows the _SIMULATE_ID should be detected too. I found the problem. The simulate id detection is done here: https://gitlab.kitware.com/cmake/cmake/blob/v3.6.1/Modules/Compiler/Clang-DetermineCompilerInternal.cmake#L12-15 and it only looks for MSVC. Grepping for SIMULATE_ID reveals that it is used for Intel and Clang but only ever detects when MSVC is simulated. All consuming code paths are interested in whether SIMULATE_ID==MSVC and otherwise assume GNU, except cmGlobalNinjaGenerator::EnableLanguage. It will take some investigation to decide how/whether to set SIMULATE_ID to GNU. Alternatively the corresponding test in the Ninja generator could be changed to look for `COMPILER_ID == Clang && SIMULATE_ID != MSVC`. -Brad From brad.king at kitware.com Wed Aug 3 09:52:41 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 3 Aug 2016 09:52:41 -0400 Subject: [cmake-developers] cmake -E capabilities [attempt 2] In-Reply-To: References: <1469105598.765.32.camel@qt.io> Message-ID: <1b6af7e5-b87b-5bc0-9c5d-2ea1b4a3be46@kitware.com> On 08/02/2016 04:48 PM, Stephen Kelly wrote: > The NewFactory methods in your patch don't return a new'd object, but > instead return static locals. The regular generators NewFactory methods > don't work that way, so you're introducing a pattern which is different to > what already exists and the commit message doesn't say why. I've revised Tobias's commits to rename NewFactory to GetFactory and explain some rationale in the commit message: Refactor extra generator registration to use factories https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a354f60c Report more information about extra generators in generator factories https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=cd52a225 I don't think the granularity is too low for these particular changes. The commit only looks large because it essentially updates the syntax used to express a table of information. Perhaps this syntax update and the main logic updates could be separated but IMO it's good enough. -Brad From brad.king at kitware.com Wed Aug 3 10:12:02 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 3 Aug 2016 10:12:02 -0400 Subject: [cmake-developers] patches from pkgsrc In-Reply-To: References: <20160728162612.9935-1-prlw1@cam.ac.uk> Message-ID: On 08/02/2016 01:08 PM, Brad King wrote: > I've applied the KWSys parts to its upstream: > > * http://review.source.kitware.com/21403 SystemInformation: Treat BSDs more uniformly > * http://review.source.kitware.com/21404 SystemInformation: Treat Solaris the same as Linux > * http://review.source.kitware.com/21405 SystemTools: Fix FileExists for some SCO OpenServer file permissions These are now in KWSys `master`. I've updated it in CMake: KWSys 2016-08-02 (3f55579d) https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3e6ec47c -Brad From prlw1 at cam.ac.uk Wed Aug 3 11:47:02 2016 From: prlw1 at cam.ac.uk (Patrick Welche) Date: Wed, 3 Aug 2016 16:47:02 +0100 Subject: [cmake-developers] patches from pkgsrc In-Reply-To: References: <20160728162612.9935-1-prlw1@cam.ac.uk> Message-ID: <20160803154702.GH26543@quartz> On Tue, Aug 02, 2016 at 01:08:23PM -0400, Brad King wrote: > I've applied the KWSys parts to its upstream: > > * http://review.source.kitware.com/21403 SystemInformation: Treat BSDs more uniformly > * http://review.source.kitware.com/21404 SystemInformation: Treat Solaris the same as Linux > * http://review.source.kitware.com/21405 SystemTools: Fix FileExists for some SCO OpenServer file permissions > > Once integrated there I can update CMake's copy. > > I left out the "Use correct cmake macro." change because it is wrong. > The SIZEOF_VOID_P macro is defined by code in CMakeLists.txt using > the value of CMAKE_SIZEOF_VOID_P. The former is a C++ preprocessor > macro. The latter is only visible in CMake code. Thanks - I'll get rid fo the CMAKE_SIZEOF_VOID_P bit. One thing: in the "Treat BSDs more uniformly" patch, there was some: +# if defined(KWSYS_SYSTEMINFORMATION_HAS_BACKTRACE) +# include +# if defined(KWSYS_SYSTEMINFORMATION_HAS_CPP_DEMANGLE) +# include +# endif +# if defined(KWSYS_SYSTEMINFORMATION_HAS_SYMBOL_LOOKUP) +# include +# endif +# endif which I don't seem to see anymore - is that part superfluous? Cheers, Patrick From brad.king at kitware.com Wed Aug 3 13:46:58 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 3 Aug 2016 13:46:58 -0400 Subject: [cmake-developers] patches from pkgsrc In-Reply-To: <20160803154702.GH26543@quartz> References: <20160728162612.9935-1-prlw1@cam.ac.uk> <20160803154702.GH26543@quartz> Message-ID: On 08/03/2016 11:47 AM, Patrick Welche wrote: > One thing: in the "Treat BSDs more uniformly" patch, there was some: > > +# if defined(KWSYS_SYSTEMINFORMATION_HAS_BACKTRACE) > +# include > +# if defined(KWSYS_SYSTEMINFORMATION_HAS_CPP_DEMANGLE) > +# include > +# endif > +# if defined(KWSYS_SYSTEMINFORMATION_HAS_SYMBOL_LOOKUP) > +# include > +# endif > +# endif > > which I don't seem to see anymore - is that part superfluous? The same block already occurs unconditionally a bit later in the file. Thanks, -Brad From aoe at google.com Wed Aug 3 14:11:45 2016 From: aoe at google.com (Chaoren Lin) Date: Wed, 3 Aug 2016 11:11:45 -0700 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> <20160802183302.GA15133@megas.kitware.com> Message-ID: > If you know /path/to/build why not just interpret relative paths with > respect to it as a heuristic? If the paths exist then use them. It's... a bit complicated. Our IDE (Android Studio) calls another build system (Gradle) which in turn calls CMake to trigger the build. So the IDE itself doesn't see the build path, only the failure output. Also, in the case of vim, the makeprg is user-supplied, so vim has no idea that it contains a path. > Another approach is to use ConvertToNinjaPath for all the build.ninja > inputs and outputs on build statements but use something other than $in > for the source file on the compilation rule. Using an "IN_ABS = ..." > variable could achieve this. Thanks, I'll try this. On Wed, Aug 3, 2016 at 5:56 AM, Brad King wrote: > On 08/02/2016 04:42 PM, Chaoren Lin wrote: > > Our IDE uses "cmake --build /path/to/build" to invoke the build, > > so that we only have to deal with CMake and not the underlying build > system. > > If you know /path/to/build why not just interpret relative paths with > respect to it as a heuristic? If the paths exist then use them. > > > Maybe I can try to fix ninja to resolve paths before sending them to the > compiler. > > The ninja build tool itself does not have enough information to make > such a transformation. CMake does though. See below. > > > Or maybe just add a flag to ConvertToNinjaPath or something. > > ConvertToNinjaPath is called from several places that don't all know > whether the file they are passing is a source file for compilation > or not. > > Another approach is to use ConvertToNinjaPath for all the build.ninja > inputs and outputs on build statements but use something other than $in > for the source file on the compilation rule. Using an "IN_ABS = ..." > variable could achieve this. > > -Brad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aoe at google.com Wed Aug 3 15:22:55 2016 From: aoe at google.com (Chaoren Lin) Date: Wed, 3 Aug 2016 12:22:55 -0700 Subject: [cmake-developers] [PATCH] Let CMake know that clang uses the GCC toolchain. In-Reply-To: <224843d8-c5b8-7d4d-6fca-727416eda120@kitware.com> References: <1469828685-28388-1-git-send-email-aoe@google.com> <65b663cd-4200-6140-651d-c630c89eb908@kitware.com> <224843d8-c5b8-7d4d-6fca-727416eda120@kitware.com> Message-ID: > It will take some investigation to decide how/whether to set SIMULATE_ID > to GNU. This seems like the right thing to do, and is more future proof than the alternative. On Wed, Aug 3, 2016 at 5:56 AM, Brad King wrote: > On 08/02/2016 06:59 PM, Chaoren Lin wrote: > >> In the gcc-like variant the CMAKE_{C,CXX}_SIMULATE_ID > >> should be detected as GNU and cause the existing code to work. > > > > Huh, it doesn't look like that variable is set automatically when using > > clang with the gcc toolchain. Is that supposed to happen? > > It should be set manually (say, in a toolchain file)? > > It is supposed to be detected automatically. Whenever Clang is detected > on Windows the _SIMULATE_ID should be detected too. I found the problem. > The simulate id detection is done here: > > > https://gitlab.kitware.com/cmake/cmake/blob/v3.6.1/Modules/Compiler/Clang-DetermineCompilerInternal.cmake#L12-15 > > and it only looks for MSVC. Grepping for SIMULATE_ID reveals that it is > used for Intel and Clang but only ever detects when MSVC is simulated. > All consuming code paths are interested in whether SIMULATE_ID==MSVC > and otherwise assume GNU, except cmGlobalNinjaGenerator::EnableLanguage. > > It will take some investigation to decide how/whether to set SIMULATE_ID > to GNU. Alternatively the corresponding test in the Ninja generator could > be changed to look for `COMPILER_ID == Clang && SIMULATE_ID != MSVC`. > > -Brad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.boeckel at kitware.com Wed Aug 3 16:12:06 2016 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Wed, 3 Aug 2016 16:12:06 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> <20160802183302.GA15133@megas.kitware.com> Message-ID: <20160803201206.GA26969@rotor.kitware.com> On Wed, Aug 03, 2016 at 11:11:45 -0700, Chaoren Lin wrote: > Also, in the case of vim, the makeprg is user-supplied, so vim has no idea > that it contains a path. What about instead of "cd path && ninja", using "ninja -C path"? --Ben From aoe at google.com Wed Aug 3 16:37:00 2016 From: aoe at google.com (Chaoren Lin) Date: Wed, 3 Aug 2016 13:37:00 -0700 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <20160803201206.GA26969@rotor.kitware.com> References: <1469828692-28544-1-git-send-email-aoe@google.com> <347b100f-f707-81fa-6b18-083adc06226e@kitware.com> <20160802183302.GA15133@megas.kitware.com> <20160803201206.GA26969@rotor.kitware.com> Message-ID: > What about instead of "cd path && ninja", using "ninja -C path"? I suppose we can also change "cmake --build path", to use "ninja -C path" instead of "cd path && ninja", but making ninja always use full paths for source files would have these benefits: 1. consistency with make (as pointed out by Brad) 2. consistency regardless of where the build directory is located (in source, under source, or out of source) 3. all three ways of invoking the build would work for IDEs (cmake --build, cd && ninja, ninja -C), so future IDE writers won't run into the sample problem no matter which way they choose On Wed, Aug 3, 2016 at 1:12 PM, Ben Boeckel wrote: > On Wed, Aug 03, 2016 at 11:11:45 -0700, Chaoren Lin wrote: > > Also, in the case of vim, the makeprg is user-supplied, so vim has no > idea > > that it contains a path. > > What about instead of "cd path && ninja", using "ninja -C path"? > > --Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Thu Aug 4 09:53:50 2016 From: brad.king at kitware.com (Brad King) Date: Thu, 4 Aug 2016 09:53:50 -0400 Subject: [cmake-developers] [PATCH] Let CMake know that clang uses the GCC toolchain. In-Reply-To: References: <1469828685-28388-1-git-send-email-aoe@google.com> <65b663cd-4200-6140-651d-c630c89eb908@kitware.com> <224843d8-c5b8-7d4d-6fca-727416eda120@kitware.com> Message-ID: On 08/03/2016 03:22 PM, Chaoren Lin wrote: >> It will take some investigation to decide how/whether to set SIMULATE_ID >> to GNU. > > This seems like the right thing to do, and is more future proof than the alternative. This patch may do it. ```patch diff --git a/Modules/Compiler/Clang-DetermineCompilerInternal.cmake b/Modules/Compiler/Clang-DetermineCompilerInternal.cmake index 08c1230..a8ef61b 100644 --- a/Modules/Compiler/Clang-DetermineCompilerInternal.cmake +++ b/Modules/Compiler/Clang-DetermineCompilerInternal.cmake @@ -12,4 +12,6 @@ set(_compiler_id_version_compute " set(_compiler_id_simulate " # if defined(_MSC_VER) # define @PREFIX at SIMULATE_ID \"MSVC\" +# elif defined(__GNUC__) +# define @PREFIX at SIMULATE_ID \"GNU\" # endif") ``` However, I remember now why we didn't do this before. For MSVC, Clang actually tries to look a lot like `cl`. It defines _MSC_VER to a version matching the compiler's standard library. OTOH for GNU, Clang is not trying to look like a specific version. It always defines __GNUC__ to 4 and __GNUC_MINOR__ to 2, regardless of what standard library is used. Essentially Clang is acting as a standalone compiler that happens to use a GNU standard library as an implementation detail. Therefore I've decided to go with the direct approach. Please try out this change: Ninja: Fix response file format for GNU-like Clang on Windows https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=e555480c Thanks, -Brad From brad.king at kitware.com Thu Aug 4 10:25:20 2016 From: brad.king at kitware.com (Brad King) Date: Thu, 4 Aug 2016 10:25:20 -0400 Subject: [cmake-developers] cmake -E capabilities [attempt 2] In-Reply-To: <1b6af7e5-b87b-5bc0-9c5d-2ea1b4a3be46@kitware.com> References: <1469105598.765.32.camel@qt.io> <1b6af7e5-b87b-5bc0-9c5d-2ea1b4a3be46@kitware.com> Message-ID: On 08/03/2016 09:52 AM, Brad King wrote: > I've revised Tobias's commits to rename NewFactory to GetFactory and > explain some rationale in the commit message: > > Refactor extra generator registration to use factories > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a354f60c > > Report more information about extra generators in generator factories > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=cd52a225 [snip] On 08/02/2016 02:30 PM, Brad King wrote: > > Once that tests cleanly I'll merge it to `master` and then we can > rebase the remaining changes on it. The above commits are now in `master`. Please rebase the rest of the topic. Please update Help/manual/cmake.1.rst to document the `-E capabilities` option. Also please add a case to Tests/RunCMake/CommandLine/RunCMakeTest.cmake to at least run `cmake -E capabilities` as a smoke test. Is there a JSON convention for documenting schema? We will need to be able to document the capabilities output format as well as the protocol formats of cmake server mode. Thanks, -Brad From konstantin at podsvirov.pro Thu Aug 4 12:31:01 2016 From: konstantin at podsvirov.pro (Konstantin Podsvirov) Date: Thu, 04 Aug 2016 19:31:01 +0300 Subject: [cmake-developers] [Review Request] Doxygen alive, but limping Message-ID: <288881470328261@web17j.yandex.ru> Hello dear CMake developers! Changes there: https://gitlab.kitware.com/cmake/cmake/merge_requests/58 -- Revive Doxygen documentation as CMake Developer Reference. This is a complete change from the point of view of the installation process. I expect that after the publication of the contents will be improved. You can preview this changes from 'CMake Master' by me. Linux: http://ifw.podsvirov.pro/cmake/cmake-master-amd64-online.run Windows: http://ifw.podsvirov.pro/cmake/cmake-master-win32-online.exe http://ifw.podsvirov.pro/cmake/cmake-master-win64-online.exe -- Regards, Konstantin Podsvirov From joubert.sy at gmail.com Thu Aug 4 13:07:41 2016 From: joubert.sy at gmail.com (Sylvain Joubert) Date: Thu, 4 Aug 2016 19:07:41 +0200 Subject: [cmake-developers] [PATCH] bash-completion: Fix cmake -E lookup Message-ID: <6adcf6c6-4a46-3e06-a3b2-efcebb6ed8e8@gmail.com> Hi, Please find attached a fix for the completion of the '-E' option. The 'sed' match was not strong enough and took into account the wrapped description text of long commands. This should now be fixed. Regards, Sylvain -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-bash-completion-Fix-cmake-E-lookup.patch Type: text/x-patch Size: 1043 bytes Desc: not available URL: From aoe at google.com Thu Aug 4 15:55:44 2016 From: aoe at google.com (Chaoren Lin) Date: Thu, 4 Aug 2016 12:55:44 -0700 Subject: [cmake-developers] [PATCH] Let CMake know that clang uses the GCC toolchain. In-Reply-To: References: <1469828685-28388-1-git-send-email-aoe@google.com> <65b663cd-4200-6140-651d-c630c89eb908@kitware.com> <224843d8-c5b8-7d4d-6fca-727416eda120@kitware.com> Message-ID: Makes sense. Thank you. On Thu, Aug 4, 2016 at 6:53 AM, Brad King wrote: > On 08/03/2016 03:22 PM, Chaoren Lin wrote: > >> It will take some investigation to decide how/whether to set SIMULATE_ID > >> to GNU. > > > > This seems like the right thing to do, and is more future proof than the > alternative. > > This patch may do it. > > ```patch > diff --git a/Modules/Compiler/Clang-DetermineCompilerInternal.cmake > b/Modules/Compiler/Clang-DetermineCompilerInternal.cmake > index 08c1230..a8ef61b 100644 > --- a/Modules/Compiler/Clang-DetermineCompilerInternal.cmake > +++ b/Modules/Compiler/Clang-DetermineCompilerInternal.cmake > @@ -12,4 +12,6 @@ set(_compiler_id_version_compute " > set(_compiler_id_simulate " > # if defined(_MSC_VER) > # define @PREFIX at SIMULATE_ID \"MSVC\" > +# elif defined(__GNUC__) > +# define @PREFIX at SIMULATE_ID \"GNU\" > # endif") > ``` > > However, I remember now why we didn't do this before. For MSVC, Clang > actually tries to look a lot like `cl`. It defines _MSC_VER to a > version matching the compiler's standard library. OTOH for GNU, Clang > is not trying to look like a specific version. It always defines > __GNUC__ to 4 and __GNUC_MINOR__ to 2, regardless of what standard > library is used. Essentially Clang is acting as a standalone compiler > that happens to use a GNU standard library as an implementation detail. > > Therefore I've decided to go with the direct approach. Please > try out this change: > > Ninja: Fix response file format for GNU-like Clang on Windows > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=e555480c > > Thanks, > -Brad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.hunger at gmail.com Fri Aug 5 05:04:14 2016 From: tobias.hunger at gmail.com (Tobias Hunger) Date: Fri, 5 Aug 2016 11:04:14 +0200 Subject: [cmake-developers] cmake -E capabilities [attempt 2] In-Reply-To: References: <1469105598.765.32.camel@qt.io> <1b6af7e5-b87b-5bc0-9c5d-2ea1b4a3be46@kitware.com> Message-ID: Hi Brad, I will add the requested changes ASAP, but I have hardly any internet connectivity at this time, so it might take a while to get the necessary patches in shape and sent. Sorry for that! Best Regards, Tobias On Thu, Aug 4, 2016 at 4:25 PM, Brad King wrote: > On 08/03/2016 09:52 AM, Brad King wrote: >> I've revised Tobias's commits to rename NewFactory to GetFactory and >> explain some rationale in the commit message: >> >> Refactor extra generator registration to use factories >> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a354f60c >> >> Report more information about extra generators in generator factories >> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=cd52a225 > [snip] > On 08/02/2016 02:30 PM, Brad King wrote: >> >> Once that tests cleanly I'll merge it to `master` and then we can >> rebase the remaining changes on it. > > The above commits are now in `master`. Please rebase the rest of > the topic. > > Please update Help/manual/cmake.1.rst to document the > `-E capabilities` option. Also please add a case to > Tests/RunCMake/CommandLine/RunCMakeTest.cmake to at least > run `cmake -E capabilities` as a smoke test. > > Is there a JSON convention for documenting schema? We will need > to be able to document the capabilities output format as well as > the protocol formats of cmake server mode. > > Thanks, > -Brad > From brad.king at kitware.com Fri Aug 5 08:55:29 2016 From: brad.king at kitware.com (Brad King) Date: Fri, 5 Aug 2016 08:55:29 -0400 Subject: [cmake-developers] [PATCH] bash-completion: Fix cmake -E lookup In-Reply-To: <6adcf6c6-4a46-3e06-a3b2-efcebb6ed8e8@gmail.com> References: <6adcf6c6-4a46-3e06-a3b2-efcebb6ed8e8@gmail.com> Message-ID: <06497920-3406-0d6b-5a1e-93f52a97741d@kitware.com> On 08/04/2016 01:07 PM, Sylvain Joubert wrote: > Please find attached a fix for the completion of the '-E' option. Thanks, applied: bash-completion: Fix cmake -E lookup https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=fe7f117a -Brad From cmake at anteru.net Fri Aug 5 16:09:28 2016 From: cmake at anteru.net (=?UTF-8?Q?Matth=c3=a4us_G._Chajdas?=) Date: Fri, 5 Aug 2016 22:09:28 +0200 Subject: [cmake-developers] [PATCH] Fix FindOpenCL on Mac OS In-Reply-To: References: Message-ID: <04313b53-8d2b-5f87-3c46-69e67d0ffad4@anteru.net> Hi, all right. Yes, that sounds like a problem with find_package_handle_standard_args, I wonder if setting the version to 0.0 by default would solve that particular problem? _DIR and _DIRS is there because that seems to be true for most packages. I'll push your patch this weekend - thanks again. Cheers, Matth?us Am 01.08.2016 um 21:04 schrieb jerry.c.t at web.de: > Hi, > >> The version is not listed as a required variable, so that's why it >> passes through. If you pass in a version into your find_package call, it >> should bail out if no version is specified. > Sadly not. The current version in master does not fail when invoked with > "find_package(OpenCL 1.2 REQUIRED)". My first email shows exactly the > output. You see that it says "Required is at least version "1.2"" > while also saying "Found OpenCL:..." while also no version was found. The > patch now fails if invoked for example with find_package(OpenCL 2.0 REQUIRED). > It seems that does not work as expected when > OpenCL_VERSION_STRING is empty. > >> I assume this was from testing, not >> because that changed something on macOS? > You are right _DIR and _DIRS are working. > I think I only changed it because for example FindGLUT uses it, for the > library the singular variant LIBRARY is used and I > don't understand the difference of _DIR and _DIRS :) > > Jerry > > On 01.08.2016 14:32, Matth?us G. Chajdas wrote: >> Hi Jerry, >> >> thanks for giving it a spin :) >> >> I don't have a Mac to test myself - as your changes are confined to >> macOS, they look safe to me. >> >> The version is not listed as a required variable, so that's why it >> passes through. If you pass in a version into your find_package call, it >> should bail out if no version is specified. >> >> I only got one minor nit-pick: Why did you change the line >> INTERFACE_INCLUDE_DIRECTORIES "${OpenCL_INCLUDE_DIR}") >> to use _DIR instead of _DIRS? I assume this was from testing, not >> because that changed something on macOS? >> >> Other than that, the patch looks good to me. I'll apply it this week. >> >> Cheers, >> Matth?us >> From aoe at google.com Fri Aug 5 19:51:56 2016 From: aoe at google.com (Chaoren Lin) Date: Fri, 5 Aug 2016 16:51:56 -0700 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: Message-ID: <1470441116-12729-1-git-send-email-aoe@google.com> From: Chaoren Lin This is consistent with the behavior of the Makefile generators. Relative paths are difficult for an IDE to parse the output of a build error. --- Source/cmNinjaTargetGenerator.cxx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Source/cmNinjaTargetGenerator.cxx b/Source/cmNinjaTargetGenerator.cxx index 855a243..5b4a55d 100644 --- a/Source/cmNinjaTargetGenerator.cxx +++ b/Source/cmNinjaTargetGenerator.cxx @@ -305,7 +305,7 @@ void cmNinjaTargetGenerator::WriteCompileRule(const std::string& lang) vars.RuleLauncher = "RULE_LAUNCH_COMPILE"; vars.CMTarget = this->GetGeneratorTarget(); vars.Language = lang.c_str(); - vars.Source = "$in"; + vars.Source = "$IN_ABS"; vars.Object = "$out"; vars.Defines = "$DEFINES"; vars.Includes = "$INCLUDES"; @@ -529,8 +529,7 @@ void cmNinjaTargetGenerator::WriteObjectBuildStatement( cmSourceFile const* source, bool writeOrderDependsTargetForTarget) { std::string const language = source->GetLanguage(); - std::string const sourceFileName = - language == "RC" ? source->GetFullPath() : this->GetSourceFilePath(source); + std::string const sourceFileName = this->GetSourceFilePath(source); std::string const objectDir = this->ConvertToNinjaPath(this->GeneratorTarget->GetSupportDirectory()); std::string const objectFileName = @@ -539,6 +538,7 @@ void cmNinjaTargetGenerator::WriteObjectBuildStatement( cmSystemTools::GetFilenamePath(objectFileName); cmNinjaVars vars; + vars["IN_ABS"] = source->GetFullPath(); vars["FLAGS"] = this->ComputeFlagsForObject(source, language); vars["DEFINES"] = this->ComputeDefines(source, language); vars["INCLUDES"] = this->GetIncludes(language); -- 2.8.0.rc3.226.g39d4020 From seblist at xwmw.org Sat Aug 6 09:42:24 2016 From: seblist at xwmw.org (Sebastian Holtermann) Date: Sat, 6 Aug 2016 15:42:24 +0200 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> Message-ID: <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> Am 02.08.2016 um 19:09 schrieb Brad King: > On 07/31/2016 03:56 PM, Sebastian Holtermann wrote: >>> I have updated the patch for CMake 3.6.0 to fix >>> https://gitlab.kitware.com/cmake/cmake/issues/16209 >> Here is an updated patch. > > Thanks. However, it is too late to restore this change for the 3.6 > series. 3.6.1 reverted the change. We can re-introduce it for 3.7 > so that it can go through another round of release candidates. Fair enough. > Please rebase on `master` and include all needed changes as if this > change were being made for the first time, including the restoration > of the tests. Ok, here is a new set of patches based on the current git master branch. I have also renamed the "same_name" test to "sameName" since camel case seems most common in CMake. -Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: cmake_3-6-1_autogen_patches_v001.tar.gz Type: application/gzip Size: 8511 bytes Desc: not available URL: From florent.castelli at gmail.com Sat Aug 6 22:54:52 2016 From: florent.castelli at gmail.com (Florent Castelli) Date: Sun, 7 Aug 2016 04:54:52 +0200 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <1470441116-12729-1-git-send-email-aoe@google.com> References: <1470441116-12729-1-git-send-email-aoe@google.com> Message-ID: <5ade5391-c553-c86c-981c-b86e90593898@gmail.com> I'd say that you shouldn't do this unless you have tested it extensively with very long command lines, making sure that there is a response file fallback if it grows too much. There are issues in CMake already on Windows with long command lines and having even longer ones is not going to help. /Orphis On 06/08/2016 01:51, Chaoren Lin via cmake-developers wrote: > From: Chaoren Lin > > This is consistent with the behavior of the Makefile generators. > Relative paths are difficult for an IDE to parse the output of a build > error. > --- > Source/cmNinjaTargetGenerator.cxx | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/Source/cmNinjaTargetGenerator.cxx b/Source/cmNinjaTargetGenerator.cxx > index 855a243..5b4a55d 100644 > --- a/Source/cmNinjaTargetGenerator.cxx > +++ b/Source/cmNinjaTargetGenerator.cxx > @@ -305,7 +305,7 @@ void cmNinjaTargetGenerator::WriteCompileRule(const std::string& lang) > vars.RuleLauncher = "RULE_LAUNCH_COMPILE"; > vars.CMTarget = this->GetGeneratorTarget(); > vars.Language = lang.c_str(); > - vars.Source = "$in"; > + vars.Source = "$IN_ABS"; > vars.Object = "$out"; > vars.Defines = "$DEFINES"; > vars.Includes = "$INCLUDES"; > @@ -529,8 +529,7 @@ void cmNinjaTargetGenerator::WriteObjectBuildStatement( > cmSourceFile const* source, bool writeOrderDependsTargetForTarget) > { > std::string const language = source->GetLanguage(); > - std::string const sourceFileName = > - language == "RC" ? source->GetFullPath() : this->GetSourceFilePath(source); > + std::string const sourceFileName = this->GetSourceFilePath(source); > std::string const objectDir = > this->ConvertToNinjaPath(this->GeneratorTarget->GetSupportDirectory()); > std::string const objectFileName = > @@ -539,6 +538,7 @@ void cmNinjaTargetGenerator::WriteObjectBuildStatement( > cmSystemTools::GetFilenamePath(objectFileName); > > cmNinjaVars vars; > + vars["IN_ABS"] = source->GetFullPath(); > vars["FLAGS"] = this->ComputeFlagsForObject(source, language); > vars["DEFINES"] = this->ComputeDefines(source, language); > vars["INCLUDES"] = this->GetIncludes(language); From irwin at beluga.phys.uvic.ca Sun Aug 7 01:39:11 2016 From: irwin at beluga.phys.uvic.ca (Alan W. Irwin) Date: Sat, 6 Aug 2016 22:39:11 -0700 (PDT) Subject: [cmake-developers] Export files in the build tree have a total path length which is unnecessarily long Message-ID: I have a simple project called test_fortran which somone else is testing for me on MinGW-w64/MSYS2, and it failed because the following file D:/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/build_tree/src_lib/CMakeFiles/Export/D_/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/install_tree/lib/cmake/test_fortran/export_test_fortran-noconfig.cmake configured (or generated) by cmake had too long a pathname for that platform. The complete cmake output for this test_fortran project was -- The Fortran compiler identification is GNU 4.9.2 -- Check for working Fortran compiler: C:/msys64/usr/bin/f95.exe -- Check for working Fortran compiler: C:/msys64/usr/bin/f95.exe -- works -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Checking whether C:/msys64/usr/bin/f95.exe supports Fortran 90 -- Checking whether C:/msys64/usr/bin/f95.exe supports Fortran 90 -- yes -- Configuring done CMake Error: cannot write to file "D:/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/build_tree/src_lib/CMakeFiles/Export/D_/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/install_tree/lib/cmake/test_fortran/export_test_fortran-noconfig.cmake": No such file or directory -- Generating done -- Build files have been written to: D:/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/build_tree (Another such file called D:/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/build_tree/src_lib/CMakeFiles/Export/D_/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/install_tree/lib/cmake/test_fortran/export_test_fortran.cmake just squeaked under the limit for that platform so was generated in the build tree OK. (Note for my test of that same project on Linux these exported files in the build tree had similar long total path lengths, but it is OK in that case because of the very long total path length on Linux.) Note that in all cases the total path length of those export files in the build tree are significantly longer than they need to be. For example, for this particular user's test of this project the build tree prefix was D:/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/build_tree so we are stuck with that part of the path length, but why is the install location of these export files (in this case D:/plplot-svn/comprehensive_test_fortran_disposeable/shared/noninteractive/install_tree/lib/cmake/test_fortran ) included (in encoded form with D: replaced by D_) in the pathname for both export files (export_test_fortran-noconfig.cmake and export_test_fortran.cmake)? That significantly increases the pathlength of these files to no purpose since the install location of these files is already known to CMake by other means. This issue is critical on platforms like MinGW-w64/MSYS2 with limited total path length so it would be good to get it addressed. If cmake developers feel the fix for this issue will require sufficient time to warrant bug triage, let me know, and I would be willing to start that process by copying the above discussion to the bugtracker. But ideally it would require only a quick fix that does not require bug triage. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ From cmake at anteru.net Sun Aug 7 13:44:29 2016 From: cmake at anteru.net (=?UTF-8?Q?Matth=c3=a4us_G._Chajdas?=) Date: Sun, 7 Aug 2016 19:44:29 +0200 Subject: [cmake-developers] [PATCH] Fix FindOpenCL on Mac OS In-Reply-To: <04313b53-8d2b-5f87-3c46-69e67d0ffad4@anteru.net> References: <04313b53-8d2b-5f87-3c46-69e67d0ffad4@anteru.net> Message-ID: Here we go: https://cmake.org/gitweb?p=stage/cmake.git;a=commit;h=48dc6343bba3b3f296d35ab060681c50f0eb8cde Thanks again for the patch! Cheers, Matth?us Am 05.08.2016 um 22:09 schrieb Matth?us G. Chajdas: > Hi, > > all right. Yes, that sounds like a problem with > find_package_handle_standard_args, I wonder if setting the version to > 0.0 by default would solve that particular problem? > > _DIR and _DIRS is there because that seems to be true for most packages. > > I'll push your patch this weekend - thanks again. > > Cheers, > Matth?us > > Am 01.08.2016 um 21:04 schrieb jerry.c.t at web.de: >> Hi, >> >>> The version is not listed as a required variable, so that's why it >>> passes through. If you pass in a version into your find_package call, it >>> should bail out if no version is specified. >> Sadly not. The current version in master does not fail when invoked with >> "find_package(OpenCL 1.2 REQUIRED)". My first email shows exactly the >> output. You see that it says "Required is at least version "1.2"" >> while also saying "Found OpenCL:..." while also no version was found. The >> patch now fails if invoked for example with find_package(OpenCL 2.0 REQUIRED). >> It seems that does not work as expected when >> OpenCL_VERSION_STRING is empty. >> >>> I assume this was from testing, not >>> because that changed something on macOS? >> You are right _DIR and _DIRS are working. >> I think I only changed it because for example FindGLUT uses it, for the >> library the singular variant LIBRARY is used and I >> don't understand the difference of _DIR and _DIRS :) >> >> Jerry >> >> On 01.08.2016 14:32, Matth?us G. Chajdas wrote: >>> Hi Jerry, >>> >>> thanks for giving it a spin :) >>> >>> I don't have a Mac to test myself - as your changes are confined to >>> macOS, they look safe to me. >>> >>> The version is not listed as a required variable, so that's why it >>> passes through. If you pass in a version into your find_package call, it >>> should bail out if no version is specified. >>> >>> I only got one minor nit-pick: Why did you change the line >>> INTERFACE_INCLUDE_DIRECTORIES "${OpenCL_INCLUDE_DIR}") >>> to use _DIR instead of _DIRS? I assume this was from testing, not >>> because that changed something on macOS? >>> >>> Other than that, the patch looks good to me. I'll apply it this week. >>> >>> Cheers, >>> Matth?us >>> > From nilsgladitz at gmail.com Mon Aug 8 08:53:08 2016 From: nilsgladitz at gmail.com (Nils Gladitz) Date: Mon, 8 Aug 2016 14:53:08 +0200 Subject: [cmake-developers] [Patch 4/5] Improved WIX support In-Reply-To: References: Message-ID: <3622e599-4d60-b835-5f0f-bf65be1653fc@gmail.com> On 07/20/2016 03:58 PM, Stuermer, Michael SP/HZA-ZSEP wrote: > Using the patchfile support I managed to implement the service installation issue I had, so the unnecessary features from the last patch are removed now. > > I tested all patches separately and hope they work now. > > looking forward for feedback, > > best regrads, > Michael I fixed a minor whitespace error (did you forget |./Utilities/SetupForDevelopment.sh ?), added a release note, extended the documentation, reworded the commit message and merged to "next" for testing. The feature itself seems to work as advertised though I am not too fond of the semantics which is why my contribution to the documentation is primarily a cautionary note. Nils | -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben.boeckel at kitware.com Mon Aug 8 09:19:39 2016 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Mon, 8 Aug 2016 09:19:39 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <5ade5391-c553-c86c-981c-b86e90593898@gmail.com> References: <1470441116-12729-1-git-send-email-aoe@google.com> <5ade5391-c553-c86c-981c-b86e90593898@gmail.com> Message-ID: <20160808131939.GA23348@megas.kitware.com> On Sun, Aug 07, 2016 at 04:54:52 +0200, Florent Castelli wrote: > I'd say that you shouldn't do this unless you have tested it extensively > with very long command lines, making sure that there is a response file > fallback if it grows too much. > There are issues in CMake already on Windows with long command lines and > having even longer ones is not going to help. Agreed. RC files compilation already has issues in some projects without absolute paths as is. https://gitlab.kitware.com/cmake/cmake/issues/16171 --Ben From brad.king at kitware.com Mon Aug 8 13:40:13 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 8 Aug 2016 13:40:13 -0400 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> Message-ID: <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> On 08/06/2016 09:42 AM, Sebastian Holtermann wrote: > Ok, here is a new set of patches based on the current git master branch. Thanks! > + outStream << "#include \"" > + << cmsys::SystemTools::ConvertToOutputPath(it->second) > + << "\"\n"; The ConvertToOutputPath method may also add its own double quotes. It is not just about slash conversion. Please use another approach to convert slashes. I dropped this patch for now. Meanwhile I've applied the rest: cmFilePathUuid: Add class to generate deterministic unique file names https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=ecb6df52 QtAutogen: Use std:: instead of ::std:: https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=06217388 QtAutogen: Allow multiple moc files with the same name https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3e2cd04b QtAutogen: Allow multiple qrc files with the same name https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=71d0308a Tests/QtAutogen: Test same moc/qrc source names in different directories https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=20f0028c and merged to `next` for testing. One of the problems reported in https://gitlab.kitware.com/cmake/cmake/issues/16209 with the previous approach was that the symbol name created by autorcc changed from what it was in CMake <= 3.5. Is it now preserved? Thanks, -Brad From brad.king at kitware.com Mon Aug 8 13:40:57 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 8 Aug 2016 13:40:57 -0400 Subject: [cmake-developers] Export files in the build tree have a total path length which is unnecessarily long In-Reply-To: References: Message-ID: On 08/07/2016 01:39 AM, Alan W. Irwin wrote: > why is the install location of these export files included in the > pathname for both export files? > That significantly increases the pathlength of these files to no purpose since the > install location of these files is already known to CMake by other means. These files are generated by the install(EXPORT) command. Their location is a staging area for "make install" to copy to the actual install destination. The install destination is used in their staging location to make sure there are no collisions. Normally the install(EXPORT) command is given a relative DESTINATION so the path is not very long. Are you giving it an absolute destination? Why? -Brad From brad.king at kitware.com Mon Aug 8 13:41:05 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 8 Aug 2016 13:41:05 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <1470441116-12729-1-git-send-email-aoe@google.com> References: <1470441116-12729-1-git-send-email-aoe@google.com> Message-ID: On 08/05/2016 07:51 PM, Chaoren Lin wrote: > - std::string const sourceFileName = > - language == "RC" ? source->GetFullPath() : this->GetSourceFilePath(source); > + std::string const sourceFileName = this->GetSourceFilePath(source); I don't think this hunk is needed anymore, correct? Thanks, -Brad From aoe at google.com Mon Aug 8 13:42:21 2016 From: aoe at google.com (Chaoren Lin) Date: Mon, 8 Aug 2016 10:42:21 -0700 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1470441116-12729-1-git-send-email-aoe@google.com> Message-ID: > I don't think this hunk is needed anymore, correct? That hunk is the opposite now, or does RC still need that hack for some reason? On Mon, Aug 8, 2016 at 10:41 AM, Brad King wrote: > On 08/05/2016 07:51 PM, Chaoren Lin wrote: > > - std::string const sourceFileName = > > - language == "RC" ? source->GetFullPath() : this->GetSourceFilePath( > source); > > + std::string const sourceFileName = this->GetSourceFilePath(source); > > I don't think this hunk is needed anymore, correct? > > Thanks, > -Brad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Mon Aug 8 13:43:12 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 8 Aug 2016 13:43:12 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <20160808131939.GA23348@megas.kitware.com> References: <1470441116-12729-1-git-send-email-aoe@google.com> <5ade5391-c553-c86c-981c-b86e90593898@gmail.com> <20160808131939.GA23348@megas.kitware.com> Message-ID: <26405a81-ec96-7b7f-da9f-0cab78a589e9@kitware.com> On 08/08/2016 09:19 AM, Ben Boeckel wrote: > On Sun, Aug 07, 2016 at 04:54:52 +0200, Florent Castelli wrote: >> I'd say that you shouldn't do this unless you have tested it extensively >> with very long command lines, making sure that there is a response file >> fallback if it grows too much. >> There are issues in CMake already on Windows with long command lines and >> having even longer ones is not going to help. > > Agreed. RC files compilation already has issues in some projects without > absolute paths as is. > > https://gitlab.kitware.com/cmake/cmake/issues/16171 RC files used absolute paths prior to this patch. The patch is for other languages. Also in out-of-source builds we already use absolute paths to files in the source tree. If there are problems with command line lengths they can be addressed separately. -Brad From brad.king at kitware.com Mon Aug 8 14:04:04 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 8 Aug 2016 14:04:04 -0400 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: References: <1470441116-12729-1-git-send-email-aoe@google.com> Message-ID: <8c2bf533-a31a-db46-d40a-537084479388@kitware.com> On 08/08/2016 01:42 PM, Chaoren Lin wrote: >> I don't think this hunk is needed anymore, correct? > > That hunk is the opposite now Oops, right. That actually fixes the existing RC bug I mentioned earlier in this thread. With your patch the use of IN_ABS breaks builds with spaces in the path. The reason is that Ninja handles quoting of paths when replacing the `$in` placeholder but does nothing special for `$IN_ABS`. CMake will have to generate the right path in the value. I've applied the patch with the appropriate modification for that: Ninja: Use full path for all source files https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=955c2a63 Thanks, -Brad From aoe at google.com Mon Aug 8 14:08:41 2016 From: aoe at google.com (Chaoren Lin) Date: Mon, 8 Aug 2016 11:08:41 -0700 Subject: [cmake-developers] [PATCH] Use full path for all source files in ninja build. In-Reply-To: <8c2bf533-a31a-db46-d40a-537084479388@kitware.com> References: <1470441116-12729-1-git-send-email-aoe@google.com> <8c2bf533-a31a-db46-d40a-537084479388@kitware.com> Message-ID: Thanks. But to address Florent's concern, are there tests that cover the windows command line limit? I guess to test if this breaks we'd need a source file whose compile command is under the limit if using relative path, but over the limit if using absolute path. On Mon, Aug 8, 2016 at 11:04 AM, Brad King wrote: > On 08/08/2016 01:42 PM, Chaoren Lin wrote: > >> I don't think this hunk is needed anymore, correct? > > > > That hunk is the opposite now > > Oops, right. That actually fixes the existing RC bug I mentioned > earlier in this thread. > > With your patch the use of IN_ABS breaks builds with spaces in the > path. The reason is that Ninja handles quoting of paths when > replacing the `$in` placeholder but does nothing special for > `$IN_ABS`. CMake will have to generate the right path in the value. > > I've applied the patch with the appropriate modification for that: > > Ninja: Use full path for all source files > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=955c2a63 > > Thanks, > -Brad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From irwin at beluga.phys.uvic.ca Mon Aug 8 14:54:54 2016 From: irwin at beluga.phys.uvic.ca (Alan W. Irwin) Date: Mon, 8 Aug 2016 11:54:54 -0700 (PDT) Subject: [cmake-developers] Export files in the build tree have a total path length which is unnecessarily long In-Reply-To: References: Message-ID: On 2016-08-08 13:40-0400 Brad King wrote: > On 08/07/2016 01:39 AM, Alan W. Irwin wrote: >> why is the install location of these export files included in the >> pathname for both export files? >> That significantly increases the pathlength of these files to no purpose since the >> install location of these files is already known to CMake by other means. > > These files are generated by the install(EXPORT) command. Their > location is a staging area for "make install" to copy to the actual > install destination. The install destination is used in their > staging location to make sure there are no collisions. > > Normally the install(EXPORT) command is given a relative DESTINATION > so the path is not very long. Are you giving it an absolute > destination? Why? PLplot (including the test_fortran subproject to be consistent with the principal plplot project) has always used absolute install locations. It sounds like I could work around the issue by moving PLplot over to relative install locations, but that is a fairly intrusive change to our build system so I am a bit reluctant to make that change. Furthermore, I doubt this problem is unique to the PLplot build system since presumably there are other projects that use absolute install locations as well because such install locations are typically well supported by CMake. The only exception to that support that I am aware of at this time is this particular install(EXPORT) case where the (encoded) absolute staging area potentially adds so much to the total length of the pathname. Is this the only case where you have a staging area in the build tree that corresponds to an install pathname? For all such staging areas, would it be straightforward to hash the paths for the cmake-generated EXPORT files in the build tree to substantially reduce their pathlengths for the absolute install location case while still avoiding name collisions? Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ From seblist at xwmw.org Mon Aug 8 16:19:39 2016 From: seblist at xwmw.org (Sebastian Holtermann) Date: Mon, 8 Aug 2016 22:19:39 +0200 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> Message-ID: <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> Am 08.08.2016 um 19:40 schrieb Brad King: > On 08/06/2016 09:42 AM, Sebastian Holtermann wrote: >> Ok, here is a new set of patches based on the current git master branch. > > Thanks! > >> + outStream << "#include \"" >> + << cmsys::SystemTools::ConvertToOutputPath(it->second) >> + << "\"\n"; > > The ConvertToOutputPath method may also add its own double quotes. > It is not just about slash conversion. Please use another approach > to convert slashes. I dropped this patch for now. Ok, I just checked http://stackoverflow.com/questions/5790161/is-the-backslash-acceptable-in-c-and-c-include-directives It is even invalid to use backslashes in an #include statements. The paths are generated with a forward slash so it should be good to just use them as they are. > Meanwhile I've applied the rest: > > cmFilePathUuid: Add class to generate deterministic unique file names > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=ecb6df52 > > QtAutogen: Use std:: instead of ::std:: > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=06217388 > > QtAutogen: Allow multiple moc files with the same name > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3e2cd04b > > QtAutogen: Allow multiple qrc files with the same name > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=71d0308a > > Tests/QtAutogen: Test same moc/qrc source names in different directories > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=20f0028c > > and merged to `next` for testing. Great, thanks. > One of the problems reported in > > https://gitlab.kitware.com/cmake/cmake/issues/16209 > > with the previous approach was that the symbol name created by autorcc > changed from what it was in CMake <= 3.5. Is it now preserved? It is back to what it was in 3.5 for the all-qrc-names-unique case. So yes, existing projects get the same symbol name they used to get. New projects that now can use non unique file names may get a symbol name based on the checksum extended file name - but that one is also length limited and should be safe. But I found another issue. The checksum string generator may generate a '-' in the file name. This is allowed for file names but not for symbol names. I have attached a small patch to fix this. -Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-QtAutogen-Replace-invalid-character-for-symbol-names.patch Type: text/x-patch Size: 923 bytes Desc: not available URL: From brad.king at kitware.com Tue Aug 9 10:11:21 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 9 Aug 2016 10:11:21 -0400 Subject: [cmake-developers] Export files in the build tree have a total path length which is unnecessarily long In-Reply-To: References: Message-ID: On 08/08/2016 02:54 PM, Alan W. Irwin wrote: > For all such staging areas, would it be straightforward to hash the > paths for the cmake-generated EXPORT files in the build tree to > substantially reduce their pathlengths for the absolute install > location case while still avoiding name collisions? Actually it looks like we already do this, but the logic forgot to account for the length of the file name. Please try this fix: install(EXPORT): Fix support for mid-length install destinations on Windows https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=c7a319ab Thanks, -Brad From brad.king at kitware.com Tue Aug 9 10:11:27 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 9 Aug 2016 10:11:27 -0400 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> Message-ID: <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> On 08/08/2016 04:19 PM, Sebastian Holtermann wrote: > It is back to what it was in 3.5 for the all-qrc-names-unique case. > So yes, existing projects get the same symbol name they used to get. Good, thanks. > New projects that now can use non unique file names > may get a symbol name based on the checksum extended file name > - but that one is also length limited and should be safe. Is this checksum user-facing in that they may need to reference such a symbol in their own code by this name? This would restrict us from ever changing the hash algorithm in the future. > But I found another issue. > The checksum string generator may generate a '-' in the file name. > This is allowed for file names but not for symbol names. > I have attached a small patch to fix this. Thanks, I squashed that in. Why does cmFilePathUuid::GetChecksumString go through so many steps of transformation? Why not just take the original hex-valued hash and use it directly (possibly truncated)? -Brad From seblist at xwmw.org Tue Aug 9 11:45:19 2016 From: seblist at xwmw.org (Sebastian Holtermann) Date: Tue, 9 Aug 2016 17:45:19 +0200 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> Message-ID: Am 09.08.2016 um 16:11 schrieb Brad King: > On 08/08/2016 04:19 PM, Sebastian Holtermann wrote: >> It is back to what it was in 3.5 for the all-qrc-names-unique case. >> So yes, existing projects get the same symbol name they used to get. > > Good, thanks. > >> New projects that now can use non unique file names >> may get a symbol name based on the checksum extended file name >> - but that one is also length limited and should be safe. > > Is this checksum user-facing in that they may need to reference such > a symbol in their own code by this name? This would restrict us from > ever changing the hash algorithm in the future. Well, it is user-facing. Though for one-binary applications the user never needs to know the the actual symbol name. Only when building libraries. It is said n "Using Resources in a Library" in http://doc.qt.io/qt-5/resources.html -- Note: The use of Q_INIT_RESOURCE() and Q_CLEANUP_RESOURCE() is not necessary when the resource is built as part of the application. -- In the non-unique-qrc-name case the checksum is hard to reproduce for a user. In this case I'd say CMake does not make any guaratees for - the actual symbol name - the symbol name to not change in the future It only guarantees the symbol to be unique within the project. If the user really needs a guranteed symbol name, e.g. for a library, he/she has to ensure that the BASENAME.qrc file name is unique within the project. In CMake 3.5 this was a requirement anyway. >> But I found another issue. >> The checksum string generator may generate a '-' in the file name. >> This is allowed for file names but not for symbol names. >> I have attached a small patch to fix this. > > Thanks, I squashed that in. > > Why does cmFilePathUuid::GetChecksumString go through so many steps > of transformation? Why not just take the original hex-valued hash > and use it directly (possibly truncated)? Base64 uses 64 different characters. Hex values only 16. Using Base64 the 14 character checksum string allows 64**14 different combinations versus 16**14 combinations when using hex. Therefore a checksum collision is less likely when using Base64. I'm not tied to Base64 but I think it provies the best uniqueness with respect to the file-length and allowed-characters limitations. -Sebastian From brad.king at kitware.com Tue Aug 9 11:52:31 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 9 Aug 2016 11:52:31 -0400 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> Message-ID: On 08/09/2016 11:45 AM, Sebastian Holtermann wrote: > In the non-unique-qrc-name case the checksum is hard to reproduce for a > user. In this case I'd say CMake does not make any guaratees for > - the actual symbol name > - the symbol name to not change in the future > It only guarantees the symbol to be unique within the project. > > If the user really needs a guranteed symbol name, e.g. for a library, > he/she has to ensure that the BASENAME.qrc file name is unique within > the project. In CMake 3.5 this was a requirement anyway. Okay. Please update the documentation accordingly. >> Why does cmFilePathUuid::GetChecksumString go through so many steps >> of transformation? Why not just take the original hex-valued hash >> and use it directly (possibly truncated)? > > Base64 uses 64 different characters. Hex values only 16. > > I'm not tied to Base64 but I think it provies the best uniqueness > with respect to the file-length and allowed-characters limitations. Okay. Please look at extending cmCryptoHash to offer access to the _Final functions to get the binary digests directly as `std::vector` or something like that. That will avoid the binary->hex->binary conversion sequence before going to Base64. Also, please explain the motivation in comments. Thanks, -Brad From seblist at xwmw.org Tue Aug 9 11:59:28 2016 From: seblist at xwmw.org (Sebastian Holtermann) Date: Tue, 9 Aug 2016 17:59:28 +0200 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> Message-ID: Am 09.08.2016 um 17:52 schrieb Brad King: > On 08/09/2016 11:45 AM, Sebastian Holtermann wrote: >> In the non-unique-qrc-name case the checksum is hard to reproduce for a >> user. In this case I'd say CMake does not make any guaratees for >> - the actual symbol name >> - the symbol name to not change in the future >> It only guarantees the symbol to be unique within the project. >> >> If the user really needs a guranteed symbol name, e.g. for a library, >> he/she has to ensure that the BASENAME.qrc file name is unique within >> the project. In CMake 3.5 this was a requirement anyway. > > Okay. Please update the documentation accordingly. > >>> Why does cmFilePathUuid::GetChecksumString go through so many steps >>> of transformation? Why not just take the original hex-valued hash >>> and use it directly (possibly truncated)? >> >> Base64 uses 64 different characters. Hex values only 16. >> >> I'm not tied to Base64 but I think it provies the best uniqueness >> with respect to the file-length and allowed-characters limitations. > > Okay. Please look at extending cmCryptoHash to offer access to > the _Final functions to get the binary digests directly as > `std::vector` or something like that. That will > avoid the binary->hex->binary conversion sequence before going > to Base64. Also, please explain the motivation in comments. > I'll look into it within the next days. -Sebastian From gjasny at googlemail.com Tue Aug 9 15:46:50 2016 From: gjasny at googlemail.com (Gregor Jasny) Date: Tue, 9 Aug 2016 21:46:50 +0200 Subject: [cmake-developers] Email feed for new CMake bugs Message-ID: <12854e94-30a9-a580-386a-43d8b9f52e07@googlemail.com> Hello, in the past (with Mantis) I found it quite helpful to see new bugs posted here on this mailing list. Is it planned to restore that functionality? Thanks, Gregor From aoe at google.com Tue Aug 9 15:47:06 2016 From: aoe at google.com (Chaoren Lin) Date: Tue, 9 Aug 2016 12:47:06 -0700 Subject: [cmake-developers] [PATCH] Fix RunCMake.Framework on case sensitive file systems. Message-ID: <1470772026-24884-1-git-send-email-aoe@google.com> The file is lowercase: Tests/RunCMake/Framework/osx.cmake --- Tests/RunCMake/Framework/RunCMakeTest.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Tests/RunCMake/Framework/RunCMakeTest.cmake b/Tests/RunCMake/Framework/RunCMakeTest.cmake index d810283..eeea6f1 100644 --- a/Tests/RunCMake/Framework/RunCMakeTest.cmake +++ b/Tests/RunCMake/Framework/RunCMakeTest.cmake @@ -20,7 +20,7 @@ unset(RunCMake_TEST_OPTIONS) set(RunCMake_TEST_BINARY_DIR ${RunCMake_BINARY_DIR}/OSXFrameworkLayout-build) set(RunCMake_TEST_NO_CLEAN 1) -set(RunCMake_TEST_OPTIONS "-DCMAKE_TOOLCHAIN_FILE=${RunCMake_SOURCE_DIR}/OSX.cmake") +set(RunCMake_TEST_OPTIONS "-DCMAKE_TOOLCHAIN_FILE=${RunCMake_SOURCE_DIR}/osx.cmake") file(REMOVE_RECURSE "${RunCMake_TEST_BINARY_DIR}") file(MAKE_DIRECTORY "${RunCMake_TEST_BINARY_DIR}") -- 2.8.0.rc3.226.g39d4020 From brad.king at kitware.com Tue Aug 9 16:01:47 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 9 Aug 2016 16:01:47 -0400 Subject: [cmake-developers] [PATCH] Fix RunCMake.Framework on case sensitive file systems. In-Reply-To: <1470772026-24884-1-git-send-email-aoe@google.com> References: <1470772026-24884-1-git-send-email-aoe@google.com> Message-ID: <85d87c77-f690-5f7c-cb38-65c9204b12ed@kitware.com> On 08/09/2016 03:47 PM, Chaoren Lin via cmake-developers wrote: > The file is lowercase: Tests/RunCMake/Framework/osx.cmake Thanks, applied: Tests: Fix RunCMake.Framework on case sensitive file systems. https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=677e73cb -Brad From gjasny at googlemail.com Tue Aug 9 16:11:08 2016 From: gjasny at googlemail.com (Gregor Jasny) Date: Tue, 9 Aug 2016 22:11:08 +0200 Subject: [cmake-developers] [PATCH] Fix RunCMake.Framework on case sensitive file systems. In-Reply-To: <85d87c77-f690-5f7c-cb38-65c9204b12ed@kitware.com> References: <1470772026-24884-1-git-send-email-aoe@google.com> <85d87c77-f690-5f7c-cb38-65c9204b12ed@kitware.com> Message-ID: On 09/08/16 22:01, Brad King wrote: > On 08/09/2016 03:47 PM, Chaoren Lin via cmake-developers wrote: >> The file is lowercase: Tests/RunCMake/Framework/osx.cmake > > Thanks, applied: > > Tests: Fix RunCMake.Framework on case sensitive file systems. > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=677e73cb Clang will get a warning for those kind of errors[1] regarding header files. Would it make sense to add similar warning-functionality to CMake? Thanks, Gregor [1] http://lists.llvm.org/pipermail/cfe-dev/2016-April/048406.html From brad.king at kitware.com Tue Aug 9 16:17:44 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 9 Aug 2016 16:17:44 -0400 Subject: [cmake-developers] Email feed for new CMake bugs In-Reply-To: <12854e94-30a9-a580-386a-43d8b9f52e07@googlemail.com> References: <12854e94-30a9-a580-386a-43d8b9f52e07@googlemail.com> Message-ID: <5ca20d79-9963-be2d-ec3a-21489714d574@kitware.com> On 08/09/2016 03:46 PM, Gregor Jasny via cmake-developers wrote: > in the past (with Mantis) I found it quite helpful to see new bugs > posted here on this mailing list. Is it planned to restore that > functionality? Not currently. GitLab has notification settings you can use to "Watch" a project, though then you will get all updates about everything instead of just new issues. It is like subscribing to another list that gets all updates about the project. GitLab notifications do not distinguish new issues/MRs from updates on existing ones. Implementing such a feed may require webhooks that send the notifications, or modification to GitLab. -Brad From brad.king at kitware.com Tue Aug 9 16:17:47 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 9 Aug 2016 16:17:47 -0400 Subject: [cmake-developers] Warn about case-insensitive mismatches (was: Fix RunCMake.Framework on case sensitive file systems.) In-Reply-To: References: <1470772026-24884-1-git-send-email-aoe@google.com> <85d87c77-f690-5f7c-cb38-65c9204b12ed@kitware.com> Message-ID: <2f00dbac-e088-097b-6a0b-af8d37b1529d@kitware.com> On 08/09/2016 04:11 PM, Gregor Jasny via cmake-developers wrote: > Clang will get a warning for those kind of errors[1] regarding header > files. Would it make sense to add similar warning-functionality to CMake? Sure, if it isn't too complicated. Thanks, -Brad From foss at grueninger.de Tue Aug 9 16:35:39 2016 From: foss at grueninger.de (=?UTF-8?Q?Christoph_Gr=c3=bcninger?=) Date: Tue, 9 Aug 2016 22:35:39 +0200 Subject: [cmake-developers] Patch: Don't emit warning when config file not found Message-ID: <5dfb692f-2d8b-fc2b-be9c-fddb09415857@grueninger.de> Dear CMake developers, I am annoyed by the lengthy and disturbing warning CMake emits when a package is not found because neither a Find*.cmake file is found nor a *config.cmake file. I always try to have zero warnings, but missing dependencies is common. Emitting a warning is too much and too noisy in my opinion. Please find attached patch, which reduces the warning to a single line with the status that the package was not found. Bye Christoph -- However, the occasional visit of success provides just the excitement an engineer needs to face work the following day. [Koichi Tanaka] -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Don-t-emit-warning-when-config-file-not-found.patch Type: text/x-patch Size: 3460 bytes Desc: not available URL: From irwin at beluga.phys.uvic.ca Tue Aug 9 19:49:40 2016 From: irwin at beluga.phys.uvic.ca (Alan W. Irwin) Date: Tue, 9 Aug 2016 16:49:40 -0700 (PDT) Subject: [cmake-developers] Export files in the build tree have a total path length which is unnecessarily long In-Reply-To: References: Message-ID: On 2016-08-09 10:11-0400 Brad King wrote: > On 08/08/2016 02:54 PM, Alan W. Irwin wrote: >> For all such staging areas, would it be straightforward to hash the >> paths for the cmake-generated EXPORT files in the build tree to >> substantially reduce their pathlengths for the absolute install >> location case while still avoiding name collisions? > > Actually it looks like we already do this, but the logic forgot > to account for the length of the file name. Please try this fix: > > install(EXPORT): Fix support for mid-length install destinations on Windows > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=c7a319ab > Hi Brad: I don't have access to Windows, but I tried this fix on Linux with a very long CMAKE_INSTALL_PREFIX (to trigger the hashing), and all seems to be well. software at raven> ~/cmake/install-c7a319a/bin/cmake -DCMAKE_INSTALL_PREFIX=$(pwd)/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/42397430947234398793874973983732987249739347293472938729487239472439782394723492374294847234927423497971498749237349723958749473934873497235987549023475/92752349572349053475945729729527234957349735209573999725497834957249524752904573975295742957349857934539709925294754973957493720274903485230252239/install_tree -DBUILD_TEST=ON .. -- The Fortran compiler identification is GNU 4.9.2 -- Check for working Fortran compiler: /usr/bin/f95 -- Check for working Fortran compiler: /usr/bin/f95 -- works -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Checking whether /usr/bin/f95 supports Fortran 90 -- Checking whether /usr/bin/f95 supports Fortran 90 -- yes -- Configuring done -- Generating done -- Build files have been written to: /home/software/plplot/HEAD/plplot.git/cmake/test_fortran/build_dir Despite that long install prefix (note the numbers have "/" embedded periodically to avoid going over the individual filename length limit on Linux), the exported files in the build-tree staging area appeared in hashed form, e.g., software at raven> pwd /home/software/plplot/HEAD/plplot.git/cmake/test_fortran/build_dir software at raven> find $(pwd) -type f |grep -i export /home/software/plplot/HEAD/plplot.git/cmake/test_fortran/build_dir/src_lib/CMakeFiles/Export/57a65d04a859ba876e044822264a11fe/export_test_fortran.cmake /home/software/plplot/HEAD/plplot.git/cmake/test_fortran/build_dir/src_lib/CMakeFiles/Export/57a65d04a859ba876e044822264a11fe/export_test_fortran-noconfig.cmake and after "make install" were installed in unhashed form (pathname length as large as 2471 for this particular test case) without issues. By the way, an install prefix of roughly 500 in length did not trigger the hashing. But I assume that lack of hashing for smaller-sized prefixes on Linux is due to some knowledge CMake has about pathname length limits on that platform, and hashing will be triggered for much smaller install prefixes on MinGW-w64/MSYS2 because of the known much smaller total pathname length limits on that platform. By the way, you already know about this from our off-list discussions, but for others here it will likely be September before that platform can be tested by the guy who originally reported the issue to me to make sure this fix solves the issue he discovered. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ From seblist at xwmw.org Wed Aug 10 06:15:05 2016 From: seblist at xwmw.org (Sebastian Holtermann) Date: Wed, 10 Aug 2016 12:15:05 +0200 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> Message-ID: <6298f722-911e-d9ba-8998-3355ad481482@xwmw.org> > Okay. Please look at extending cmCryptoHash to offer access to > the _Final functions to get the binary digests directly as > `std::vector` or something like that. That will > avoid the binary->hex->binary conversion sequence before going > to Base64. Also, please explain the motivation in comments. These patches change the private abstract interface of cmCryptoHash to return std::vector instead of std::string. Conversion to std::string is handled by a static public method. -Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: cmake_3-6-1_cmCryptoHash_patches_v001.tar.gz Type: application/gzip Size: 4387 bytes Desc: not available URL: From brad.king at kitware.com Wed Aug 10 09:17:01 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 10 Aug 2016 09:17:01 -0400 Subject: [cmake-developers] Patch: Don't emit warning when config file not found In-Reply-To: <5dfb692f-2d8b-fc2b-be9c-fddb09415857@grueninger.de> References: <5dfb692f-2d8b-fc2b-be9c-fddb09415857@grueninger.de> Message-ID: <25936235-f6e3-2b0a-5e42-2e0ea92413f3@kitware.com> On 08/09/2016 04:35 PM, Christoph Gr?ninger wrote: > I am annoyed by the lengthy and disturbing warning CMake emits when a > package is not found because neither a Find*.cmake file is found nor a > *config.cmake file. I always try to have zero warnings, but missing > dependencies is common. Emitting a warning is too much and too noisy in > my opinion. > > Please find attached patch, which reduces the warning to a single line > with the status that the package was not found. We went through many iterations on the wording of that message in response to user feedback before settling on what is currently there. When someone wants a package to be found they need a good explanation when it is not and hints about how to resolve the problem. If a dependency is optional and it is not a problem to be missing then the project can use the find_package QUIET option and then optionally add its own message() about the missing package. -Brad From brad.king at kitware.com Wed Aug 10 13:33:42 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 10 Aug 2016 13:33:42 -0400 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: <6298f722-911e-d9ba-8998-3355ad481482@xwmw.org> References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> <6298f722-911e-d9ba-8998-3355ad481482@xwmw.org> Message-ID: On 08/10/2016 06:15 AM, Sebastian Holtermann wrote: > These patches change the private abstract interface of cmCryptoHash to > return std::vector instead of std::string. > Conversion to std::string is handled by a static public method. Nice. I've added this to the topic, now merged for testing here: Merge topic 'autogen-same-name' into next https://cmake.org/gitweb?p=cmake.git;a=commit;h=df237068 Now all this topic is missing is an update to the documentation to explain the behavior w.r.t. duplicate file names versus the symbol names. Thanks, -Brad From konstantin at podsvirov.pro Thu Aug 11 11:36:02 2016 From: konstantin at podsvirov.pro (Konstantin Podsvirov) Date: Thu, 11 Aug 2016 18:36:02 +0300 Subject: [cmake-developers] [Review Request] Doxygen alive, but limping In-Reply-To: <288881470328261@web17j.yandex.ru> References: <288881470328261@web17j.yandex.ru> Message-ID: <39611470929762@web30g.yandex.ru> Hello dear CMake developers! 04.08.2016, 19:31, "Konstantin Podsvirov" : > > Changes there: > > https://gitlab.kitware.com/cmake/cmake/merge_requests/58 > Now forced updated. > -- > > Revive Doxygen documentation as CMake Developer Reference. > > This is a complete change from the point of view of the installation process. > > I expect that after the publication of the contents will be improved. > > You can preview this changes from 'CMake Master' by me. > > Linux: > > http://ifw.podsvirov.pro/cmake/cmake-master-amd64-online.run > > Windows: > > http://ifw.podsvirov.pro/cmake/cmake-master-win32-online.exe > > http://ifw.podsvirov.pro/cmake/cmake-master-win64-online.exe > Also available online documentation: http://ifw.podsvirov.pro/cmake/doc/sphinx http://ifw.podsvirov.pro/cmake/doc/doxygen -- Regards, Konstantin Podsvirov From Stephen.Sorley at jhuapl.edu Thu Aug 11 15:37:21 2016 From: Stephen.Sorley at jhuapl.edu (Sorley, Stephen L.) Date: Thu, 11 Aug 2016 19:37:21 +0000 Subject: [cmake-developers] Bugfix: Default behavior of CUDA_USE_STATIC_CUDA_RUNTIME Message-ID: Recent changes to FindCUDA.cmake (commit 7229ae7) broke the documented behavior of the CUDA_USE_STATIC_CUDA_RUNTIME option. Per the documentation, this option should be enabled by default if libcudart_static is present in the selected version of CUDA, and silently disabled if it's not. After commit 7229ae7, the option is disabled by default instead. I've attached a small patch (based on master) that fixes the problem. Thanks! Stephen Sorley -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Restore-default-behavior-of-CUDA_USE_STATIC_CUDA_RUN.patch Type: application/octet-stream Size: 1648 bytes Desc: 0001-Restore-default-behavior-of-CUDA_USE_STATIC_CUDA_RUN.patch URL: From Stephen.Sorley at jhuapl.edu Thu Aug 11 16:10:45 2016 From: Stephen.Sorley at jhuapl.edu (Sorley, Stephen L.) Date: Thu, 11 Aug 2016 20:10:45 +0000 Subject: [cmake-developers] Bugfix: Default behavior of CUDA_USE_STATIC_CUDA_RUNTIME Message-ID: <42e2f59261b24e49a98b8b206f23955d@APLEX03.dom1.jhuapl.edu> The previous fix uncovered another bug from commit 7229ae7. That commit dropped librt from the link line when using static linking with CUDA 7.0 or newer on Linux. However, librt is still required on the link line in newer CUDA versions, I get linker errors without it. I've attached a second patch to fix this extra bug. Thanks! Stephen Sorley From: Sorley, Stephen L. Sent: Thursday, August 11, 2016 3:37 PM To: cmake-developers at cmake.org Subject: Bugfix: Default behavior of CUDA_USE_STATIC_CUDA_RUNTIME Recent changes to FindCUDA.cmake (commit 7229ae7) broke the documented behavior of the CUDA_USE_STATIC_CUDA_RUNTIME option. Per the documentation, this option should be enabled by default if libcudart_static is present in the selected version of CUDA, and silently disabled if it's not. After commit 7229ae7, the option is disabled by default instead. I've attached a small patch (based on master) that fixes the problem. Thanks! Stephen Sorley -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-for-missing-librt-on-Linux-when-using-static-cud.patch Type: application/octet-stream Size: 1439 bytes Desc: 0001-Fix-for-missing-librt-on-Linux-when-using-static-cud.patch URL: From costean_adrian at yahoo.com Fri Aug 12 03:17:18 2016 From: costean_adrian at yahoo.com (Costean Adrian) Date: Fri, 12 Aug 2016 07:17:18 +0000 (UTC) Subject: [cmake-developers] Integrate tools with cmake References: <1657600095.12864829.1470986238438.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <1657600095.12864829.1470986238438.JavaMail.yahoo@mail.yahoo.com> Hello, I am currently developing some projects which involves a build system and some tools. I would like to know what are the possibilities when it comes to integrate other tools in the cmake build system? How can that be done and what are the obstacles for achieving that? Thanks and regards, Adrian C. From nilsgladitz at gmail.com Fri Aug 12 03:41:30 2016 From: nilsgladitz at gmail.com (Nils Gladitz) Date: Fri, 12 Aug 2016 09:41:30 +0200 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: References: Message-ID: On 07/20/2016 03:58 PM, Stuermer, Michael SP/HZA-ZSEP wrote: > Using the patchfile support I managed to implement the service installation issue I had, so the unnecessary features from the last patch are removed now. > > I tested all patches separately and hope they work now. > > looking forward for feedback, > > best regrads, > Michael > Patch 5 seems to implement patching of FeatureRef rather than the original Feature elements. I am not sure if this is what you intended but this could be error prone given that there could in theory be any number (0-n) FeatureRef elements for any corresponding Feature element. Nils From smspillaz at gmail.com Fri Aug 12 04:28:04 2016 From: smspillaz at gmail.com (Sam Spilsbury) Date: Fri, 12 Aug 2016 16:28:04 +0800 Subject: [cmake-developers] Integrate tools with cmake In-Reply-To: <1657600095.12864829.1470986238438.JavaMail.yahoo@mail.yahoo.com> References: <1657600095.12864829.1470986238438.JavaMail.yahoo.ref@mail.yahoo.com> <1657600095.12864829.1470986238438.JavaMail.yahoo@mail.yahoo.com> Message-ID: Hi there, Depends on what you mean by "tools". If you mean running some sort of static analysis over your code, there's a few ways to do it. I prefer to use add_custom_command which generates a new makefile based on some output stampfile. You can make the stampfile depend on the input file that you're analysing and add the stampfile as a source-level dependency to your built library or binary. This way, the tool will be re-run every time you modify the file. For instance: add_custom_command (OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${SOURCE_NAME}.stamp COMMAND my-tool ${SOURCE_NAME} COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_CURRENT_BINARY_DIR}/${SOURCE_NAME}.stamp DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${SOURCE_NAME} COMMENT "Running my-tool on ${SOURCE_NAME}") [self-promotion]: I was doing this for quite a few tools, so I wrote a module to quickly define functions that allow you to run various tools over source files: https://github.com/polysquare/tooling-cmake-util/blob/master/PolysquareToolingUtil.cmake It has some dependencies which you can either install using conan or just add them as git submodules and set CMAKE_MODULE_PATH appropriately. You can contact me on this email address if you need any help. Thanks, Sam. On Fri, Aug 12, 2016 at 3:17 PM, Costean Adrian via cmake-developers wrote: > Hello, > I am currently developing some projects which involves a build system and some tools. I would like to know what are the possibilities when it comes to integrate other tools in the cmake build system? How can that be done and what are the obstacles for achieving that? > Thanks and regards, > Adrian C. > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -- Sam Spilsbury From michael.stuermer at schaeffler.com Fri Aug 12 05:50:38 2016 From: michael.stuermer at schaeffler.com (Stuermer, Michael SP/HZA-ZSEP) Date: Fri, 12 Aug 2016 09:50:38 +0000 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: References: Message-ID: > -----Original Message----- > From: Nils Gladitz [mailto:nilsgladitz at gmail.com] > Sent: Friday, August 12, 2016 9:42 AM > To: Stuermer, Michael SP/HZA-ZSEP; CMake Developers > Subject: Re: [cmake-developers] [Patch 5/5] Improved WIX support > > On 07/20/2016 03:58 PM, Stuermer, Michael SP/HZA-ZSEP wrote: > > > Using the patchfile support I managed to implement the service installation > issue I had, so the unnecessary features from the last patch are removed > now. > > > > I tested all patches separately and hope they work now. > > > > looking forward for feedback, > > > > best regrads, > > Michael > > > > Patch 5 seems to implement patching of FeatureRef rather than the original > Feature elements. > I am not sure if this is what you intended but this could be error prone given > that there could in theory be any number (0-n) FeatureRef elements for any > corresponding Feature element. > > Nils The intention was to be able to add custom components that are added as extra .wxs source files to certain features. If there are more convenient ways to do this I will be happy to change the implementation or adapt my WIX project. But so far this seemed to be a very easy and intuitive solution: the additional component references are added in the same place where all other component references are added as well. Michael From nilsgladitz at gmail.com Fri Aug 12 07:49:50 2016 From: nilsgladitz at gmail.com (Nils Gladitz) Date: Fri, 12 Aug 2016 13:49:50 +0200 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: References: Message-ID: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> On 08/12/2016 11:50 AM, Stuermer, Michael SP/HZA-ZSEP wrote: > >> Patch 5 seems to implement patching of FeatureRef rather than the original >> Feature elements. >> I am not sure if this is what you intended but this could be error prone given >> that there could in theory be any number (0-n) FeatureRef elements for any >> corresponding Feature element. >> >> Nils > The intention was to be able to add custom components that are added as extra .wxs source files to certain features. If there are more convenient ways to do this I will be happy to change the implementation or adapt my WIX project. But so far this seemed to be a very easy and intuitive solution: the additional component references are added in the same place where all other component references are added as well. I understand the general intention but not why you elected to patch FeatureRef elements instead of the Feature elements themselves. This would not be any more convenient but would certainly match expectations and be less ill defined. e.g. when I specify a patch for a Feature I expect that the Feature with the given ID gets patched. Arguments against patching a FeatureRef instead are: - There can be n FeatureRef elements for any Feature element in which case it would not be obvious if the patch should be applied to one (which?) or all of these - While similar FeatureRef and Feature don't take the same Child elements - You can already define your own FeatureRef elements (referencing any of the pre-existing Feature elements if wanted) without having to use the patch mechanism Nils From forumer at smartmobili.com Fri Aug 12 09:05:36 2016 From: forumer at smartmobili.com (Forumer 4umer) Date: Fri, 12 Aug 2016 15:05:36 +0200 Subject: [cmake-developers] toolset Clang with Microsoft CodeGen (v140_clang_c2) debug is broken Message-ID: <2c7ec64cd11d8d156d1c11e2fa72295f@smartmobili.com> Hi, I have tested cmake 3.6.1 with the last release of the clang platform by Microsoft (v140_clang_c2) and nothing has changed. By default cmake generates wrong projects with this toolset. So could we fix it step by step: In the next release of cmake could you please remove the -gline-tables-only (and maybe all the additional arguments you provide because Microsoft don't add them when using Visual to generate projects) Then if you have time please Disable Optimizations in Debug and finally please set DebugInformationFormat for debug configurations. For now I have to manually fix all my projects everytime I regenerate them and it's not very funny. Thanks From forumer at smartmobili.com Fri Aug 12 08:59:38 2016 From: forumer at smartmobili.com (Forumer 4umer) Date: Fri, 12 Aug 2016 14:59:38 +0200 Subject: [cmake-developers] toolset Clang 3.7 with Microsoft CodeGen (v140_clang_3_7) debug is broken In-Reply-To: References: <4cc99846ec062f9c697bbcf5cd804751@smartmobili.com> <57279F63.300@kitware.com> Message-ID: <4c3bcdcceae85c65468faefce1978a43@smartmobili.com> Le 2016-05-02 20:48, forumer at xxxx.com a ?crit?: > Le 2016-05-02 20:41, Brad King a ?crit?: >> On 05/02/2016 02:17 PM, forumer at xxxxx.com wrote: >>> Don't know if it's the best place to report it but I am testing >>> nightly >>> builds(3.5.20160429) to generate some projects >>> for Visual Studio 2015 and the clang toolset now provided by >>> Microsoft. >> >> Yes, this is the right place to report it, especially since this is >> recent development. For reference, support for this toolset was added >> in a topic merged here: >> >> Merge topic 'vs-clang-cl' >> https://cmake.org/gitweb?p=cmake.git;a=commit;h=612a8b3b >> >> based on discussion in this thread: >> >> [PATCH SET] Support of Clang/C2 compiler. >> >> http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15902 >> >>> I have found some changes that seems to be necessary to be able to >>> debug, here are the changes I have found: >>> >>> 1) DebugInformationFormat should be set FullDebug, AdditionalOptions >>> MUST BE removed and optimizations must be disabled >>> >>> >>> ... REMOVE AdditionalOptions ... >>> FullDebug >>> ... >>> Disabled >>> false >>> >>> >>> >>> 2)GenerateDebugInformation must be set to true and >>> ProgramDataBaseFile >>> must be empty because clang doesn't use pdb for now >>> >>> ... >>> true >>> ... >>> >>> >>> >>> >> >> Mariusz, please take a look at this. >> >> See also this hack: >> >> VS: Fix VS 2015 .vcxproj debug setting for older toolsets >> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=dc422d27 >> >> Basically we need to have a different flag table for each possible >> toolset. Also it looks like additional special handling may be >> needed for PDBs and perhaps other limitations of the Clang/C2 >> toolchain. >> >> Thanks, >> -Brad > > Just a small update about what I wrote regarding AdditionalOptions, I > found that it's not necessary to remove the whole line but only the > -gline-tables-only parameter. > Hi, I have tested cmake 3.6.1 with the last release of the clang platform by Microsoft (v140_clang_c2) and nothing has changed. By default cmake generates wrong projects with this toolset. So could we fix it step by step: In the next release of cmake could you please remove the -gline-tables-only (and maybe all the additional arguments you provide because Microsoft don't add them when using Visual to generate projects) Then if you have time please Disable Optimizations in Debug and finally please set DebugInformationFormat for debug configurations. For now I have to manually fix all my projects everytime I regenerate them and it's not very funny. Thanks From forumer at smartmobili.com Fri Aug 12 09:14:02 2016 From: forumer at smartmobili.com (Forumer 4umer) Date: Fri, 12 Aug 2016 15:14:02 +0200 Subject: [cmake-developers] toolset Clang with Microsoft CodeGen (v140_clang_c2) debug is broken In-Reply-To: <2c7ec64cd11d8d156d1c11e2fa72295f@smartmobili.com> References: <2c7ec64cd11d8d156d1c11e2fa72295f@smartmobili.com> Message-ID: And if you need more information please look at the thread entitled "toolset Clang 3.7 with Microsoft CodeGen (v140_clang_3_7) debug is broken" a few months ago. From brad.king at kitware.com Fri Aug 12 09:17:43 2016 From: brad.king at kitware.com (Brad King) Date: Fri, 12 Aug 2016 09:17:43 -0400 Subject: [cmake-developers] toolset Clang 3.7 with Microsoft CodeGen (v140_clang_3_7) debug is broken In-Reply-To: <4c3bcdcceae85c65468faefce1978a43@smartmobili.com> References: <4cc99846ec062f9c697bbcf5cd804751@smartmobili.com> <57279F63.300@kitware.com> <4c3bcdcceae85c65468faefce1978a43@smartmobili.com> Message-ID: <85e7e0c6-f868-8d5e-6c3b-f408e5840283@kitware.com> On 08/12/2016 08:59 AM, Forumer 4umer wrote: > I have tested cmake 3.6.1 with the last release of the clang platform by > Microsoft (v140_clang_c2) and nothing has changed. No one is actively working on support for Clang toolsets currently. The experimental support in 3.6 was contributed but no one has maintained it. There is an issue covering the main problem: https://gitlab.kitware.com/cmake/cmake/issues/16153 Discussion there has links to some of the other problems. -Brad From brad.king at kitware.com Fri Aug 12 09:35:34 2016 From: brad.king at kitware.com (Brad King) Date: Fri, 12 Aug 2016 09:35:34 -0400 Subject: [cmake-developers] Bugfix: Default behavior of CUDA_USE_STATIC_CUDA_RUNTIME In-Reply-To: References: Message-ID: Hi Stephen, On 08/11/2016 03:37 PM, Sorley, Stephen L. wrote: > Recent changes to FindCUDA.cmake (commit 7229ae7) broke Thanks for these two fixes. I've applied them: FindCUDA: Restore default behavior of CUDA_USE_STATIC_CUDA_RUNTIME https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=aab1f6e9 FindCUDA: Fix missing librt on Linux when using static cuda runtime. https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=1aa5c1be James, do these look good to you? Thanks, -Brad From brad.king at kitware.com Fri Aug 12 11:13:25 2016 From: brad.king at kitware.com (Brad King) Date: Fri, 12 Aug 2016 11:13:25 -0400 Subject: [cmake-developers] Android Support Message-ID: <3144149b-a4f9-0f3b-0a85-9f1eefc2b0ad@kitware.com> Hi Folks, I've implemented native support for cross-compiling to Android with CMake using either an Android NDK or an Android Standalone Toolchain. I plan to include this in the CMake 3.7 release. Please see the MR here: https://gitlab.kitware.com/cmake/cmake/merge_requests/62 This enables building with simple toolchain files, or even without one: $ cmake ../src -DCMAKE_SYSTEM_NAME=Android -DCMAKE_ANDROID_NDK=/path/to/ndk Note that interface compatibility with popular third-party Android toolchain files is a non-goal of this work (though we should not break them too much). Instead I'd like to make the support feel native within CMake. For example, the Android API level is naturally specified by CMAKE_SYSTEM_VERSION. If you're interested in this support please fetch the MR branch and try it out. The Help/manual/cmake-toolchains.7.rst manual has been updated with documentation about how to use this. -Brad From Robert.Goulet at autodesk.com Fri Aug 12 11:26:33 2016 From: Robert.Goulet at autodesk.com (Robert Goulet) Date: Fri, 12 Aug 2016 15:26:33 +0000 Subject: [cmake-developers] Android Support In-Reply-To: <3144149b-a4f9-0f3b-0a85-9f1eefc2b0ad@kitware.com> References: <3144149b-a4f9-0f3b-0a85-9f1eefc2b0ad@kitware.com> Message-ID: That's super awesome Brad! Looking forward to try this out! -----Original Message----- From: cmake-developers [mailto:cmake-developers-bounces at cmake.org] On Behalf Of Brad King Sent: Friday, August 12, 2016 11:13 AM To: cmake-developers at cmake.org Subject: [cmake-developers] Android Support Hi Folks, I've implemented native support for cross-compiling to Android with CMake using either an Android NDK or an Android Standalone Toolchain. I plan to include this in the CMake 3.7 release. Please see the MR here: https://gitlab.kitware.com/cmake/cmake/merge_requests/62 This enables building with simple toolchain files, or even without one: $ cmake ../src -DCMAKE_SYSTEM_NAME=Android -DCMAKE_ANDROID_NDK=/path/to/ndk Note that interface compatibility with popular third-party Android toolchain files is a non-goal of this work (though we should not break them too much). Instead I'd like to make the support feel native within CMake. For example, the Android API level is naturally specified by CMAKE_SYSTEM_VERSION. If you're interested in this support please fetch the MR branch and try it out. The Help/manual/cmake-toolchains.7.rst manual has been updated with documentation about how to use this. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers From rcdailey.lists at gmail.com Fri Aug 12 15:59:31 2016 From: rcdailey.lists at gmail.com (Robert Dailey) Date: Fri, 12 Aug 2016 14:59:31 -0500 Subject: [cmake-developers] Need ideas/opinions on third party library management Message-ID: Hello, There is an internal C++ product at the company I work for which I have written a series of CMake scripts for. This project actually has dependencies on several open source libraries, such as boost, freetype, openssl, etc. Right now what we do is build each of these third party libraries *by hand*, once for every platform we support (Windows, Linux x86, Android NDK). Then we stuff the includes (headers) and libraries (static/shared) in a submodule and the primary code base's CMake scripts pull them in as interface targets. This works well and is light-weight but is a pain when upgrading or changing libraries. It's a pain because if I want to upgrade boost, I have to build it up to 6 times (once for each platform and once for each configuration). I've been thinking of a different approach for a while. I've done some toying around with the "Super Build" concept, where I have a separate CMake project that does nothing but use the ExternalProject module to build libraries in real time along with our project. So the order of operations would be as follows (for our automated build server): 1. Clone our "Third Party" repository 2. Use CMake to generate & build the "Super Build" project (this builds boost, openssl, freetype, etc for the current platform). 3. Clone the main code base's repository 4. Use CMake to generate & build, using find_package() to refer to interface targets exported by those third party libraries built in step 2 Obviously this will make builds take significantly longer, because we're constantly rebuilding the same third party libraries over and over again. However, it virtually eliminates the maintenance burden for third party libraries because they are built inherently with everything else. Note that I can't refer to pre-built libraries in our build environments because we need very specific control over the versions of our libraries as well as the toolchains that were used to build them. Also we may specifically build our libraries a certain way (such as boost). For this reason we do not rely on our external environment or external package managers to fulfill third party dependencies, like most open source projects do on Linux for example. Does this "Super Build" approach sound like a better idea? What other options are available? The downside with the "Super Build" solution is that it will become very difficult to make the transition between building third party and building our code base seamless. I can't do both in the same generate step because find_package() can't be called until the libraries are built & installed. From elizabeth.fischer at columbia.edu Fri Aug 12 20:12:03 2016 From: elizabeth.fischer at columbia.edu (Elizabeth A. Fischer) Date: Fri, 12 Aug 2016 20:12:03 -0400 Subject: [cmake-developers] [CMake] Need ideas/opinions on third party library management In-Reply-To: References: Message-ID: > > This is what Spack and other meta builders do. I don't think CMake is the > best place to do it, for a number of reasons. I would not try to re-invent > the wheel here. > See http://github. com/llnl/spack > -- Elizabeth -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcdailey.lists at gmail.com Sat Aug 13 19:00:46 2016 From: rcdailey.lists at gmail.com (Robert Dailey) Date: Sat, 13 Aug 2016 18:00:46 -0500 Subject: [cmake-developers] [CMake] Need ideas/opinions on third party library management In-Reply-To: References: Message-ID: I did some brief digging into spack, and it doesn't look like it supports Windows. All I see are shell scripts and the documentation uses POSIX. If I'm going to use a package manager, it needs to be able to support Android (ARM), Windows, and Linux. I have specific toolchains that I'll need the package manager to use for each of these platforms, assuming it compiles these libraries. It also needs to be capable of cross-compiling (in the case of ARM toolchain in the Android NDK). I mean, we can't really call it "reinventing the wheel" if my requirements are so specific that no current tooling can support it. If you have any info on spack related to my requirements please let me know. It looks promising, but so far doesn't seem like it will work out. On Fri, Aug 12, 2016 at 3:41 PM, Elizabeth A. Fischer wrote: > This is what Spack and other meta builders do. See http://github. > com/llnl/spack > > On Aug 12, 2016 3:59 PM, "Robert Dailey" wrote: >> >> Hello, >> >> There is an internal C++ product at the company I work for which I >> have written a series of CMake scripts for. This project actually has >> dependencies on several open source libraries, such as boost, >> freetype, openssl, etc. >> >> Right now what we do is build each of these third party libraries *by >> hand*, once for every platform we support (Windows, Linux x86, Android >> NDK). Then we stuff the includes (headers) and libraries >> (static/shared) in a submodule and the primary code base's CMake >> scripts pull them in as interface targets. >> >> This works well and is light-weight but is a pain when upgrading or >> changing libraries. It's a pain because if I want to upgrade boost, I >> have to build it up to 6 times (once for each platform and once for >> each configuration). >> >> I've been thinking of a different approach for a while. I've done some >> toying around with the "Super Build" concept, where I have a separate >> CMake project that does nothing but use the ExternalProject module to >> build libraries in real time along with our project. So the order of >> operations would be as follows (for our automated build server): >> >> 1. Clone our "Third Party" repository >> 2. Use CMake to generate & build the "Super Build" project (this >> builds boost, openssl, freetype, etc for the current platform). >> 3. Clone the main code base's repository >> 4. Use CMake to generate & build, using find_package() to refer to >> interface targets exported by those third party libraries built in >> step 2 >> >> Obviously this will make builds take significantly longer, because >> we're constantly rebuilding the same third party libraries over and >> over again. However, it virtually eliminates the maintenance burden >> for third party libraries because they are built inherently with >> everything else. >> >> Note that I can't refer to pre-built libraries in our build >> environments because we need very specific control over the versions >> of our libraries as well as the toolchains that were used to build >> them. Also we may specifically build our libraries a certain way (such >> as boost). For this reason we do not rely on our external environment >> or external package managers to fulfill third party dependencies, like >> most open source projects do on Linux for example. >> >> Does this "Super Build" approach sound like a better idea? What other >> options are available? The downside with the "Super Build" solution is >> that it will become very difficult to make the transition between >> building third party and building our code base seamless. I can't do >> both in the same generate step because find_package() can't be called >> until the libraries are built & installed. >> -- >> >> Powered by www.kitware.com >> >> Please keep messages on-topic and check the CMake FAQ at: >> http://www.cmake.org/Wiki/CMake_FAQ >> >> Kitware offers various services to support the CMake community. For more >> information on each offering, please visit: >> >> CMake Support: http://cmake.org/cmake/help/support.html >> CMake Consulting: http://cmake.org/cmake/help/consulting.html >> CMake Training Courses: http://cmake.org/cmake/help/training.html >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Follow this link to subscribe/unsubscribe: >> http://public.kitware.com/mailman/listinfo/cmake From elizabeth.fischer at columbia.edu Sat Aug 13 19:42:35 2016 From: elizabeth.fischer at columbia.edu (Elizabeth A. Fischer) Date: Sat, 13 Aug 2016 19:42:35 -0400 Subject: [cmake-developers] [CMake] Need ideas/opinions on third party library management In-Reply-To: References: Message-ID: I would look into Anaconda, which does work for Windows. Its version management is not as well developed as Spack, but it's more cross-platform. Auto-builders are just coming into their own, it's a brave new world. I expect things to be more complete in a few years. -- Elizabeth On Sat, Aug 13, 2016 at 7:00 PM, Robert Dailey wrote: > I did some brief digging into spack, and it doesn't look like it > supports Windows. All I see are shell scripts and the documentation > uses POSIX. > > If I'm going to use a package manager, it needs to be able to support > Android (ARM), Windows, and Linux. I have specific toolchains that > I'll need the package manager to use for each of these platforms, > assuming it compiles these libraries. It also needs to be capable of > cross-compiling (in the case of ARM toolchain in the Android NDK). > > I mean, we can't really call it "reinventing the wheel" if my > requirements are so specific that no current tooling can support it. > If you have any info on spack related to my requirements please let me > know. It looks promising, but so far doesn't seem like it will work > out. > > On Fri, Aug 12, 2016 at 3:41 PM, Elizabeth A. Fischer > wrote: > > This is what Spack and other meta builders do. See http://github. > > com/llnl/spack > > > > On Aug 12, 2016 3:59 PM, "Robert Dailey" > wrote: > >> > >> Hello, > >> > >> There is an internal C++ product at the company I work for which I > >> have written a series of CMake scripts for. This project actually has > >> dependencies on several open source libraries, such as boost, > >> freetype, openssl, etc. > >> > >> Right now what we do is build each of these third party libraries *by > >> hand*, once for every platform we support (Windows, Linux x86, Android > >> NDK). Then we stuff the includes (headers) and libraries > >> (static/shared) in a submodule and the primary code base's CMake > >> scripts pull them in as interface targets. > >> > >> This works well and is light-weight but is a pain when upgrading or > >> changing libraries. It's a pain because if I want to upgrade boost, I > >> have to build it up to 6 times (once for each platform and once for > >> each configuration). > >> > >> I've been thinking of a different approach for a while. I've done some > >> toying around with the "Super Build" concept, where I have a separate > >> CMake project that does nothing but use the ExternalProject module to > >> build libraries in real time along with our project. So the order of > >> operations would be as follows (for our automated build server): > >> > >> 1. Clone our "Third Party" repository > >> 2. Use CMake to generate & build the "Super Build" project (this > >> builds boost, openssl, freetype, etc for the current platform). > >> 3. Clone the main code base's repository > >> 4. Use CMake to generate & build, using find_package() to refer to > >> interface targets exported by those third party libraries built in > >> step 2 > >> > >> Obviously this will make builds take significantly longer, because > >> we're constantly rebuilding the same third party libraries over and > >> over again. However, it virtually eliminates the maintenance burden > >> for third party libraries because they are built inherently with > >> everything else. > >> > >> Note that I can't refer to pre-built libraries in our build > >> environments because we need very specific control over the versions > >> of our libraries as well as the toolchains that were used to build > >> them. Also we may specifically build our libraries a certain way (such > >> as boost). For this reason we do not rely on our external environment > >> or external package managers to fulfill third party dependencies, like > >> most open source projects do on Linux for example. > >> > >> Does this "Super Build" approach sound like a better idea? What other > >> options are available? The downside with the "Super Build" solution is > >> that it will become very difficult to make the transition between > >> building third party and building our code base seamless. I can't do > >> both in the same generate step because find_package() can't be called > >> until the libraries are built & installed. > >> -- > >> > >> Powered by www.kitware.com > >> > >> Please keep messages on-topic and check the CMake FAQ at: > >> http://www.cmake.org/Wiki/CMake_FAQ > >> > >> Kitware offers various services to support the CMake community. For more > >> information on each offering, please visit: > >> > >> CMake Support: http://cmake.org/cmake/help/support.html > >> CMake Consulting: http://cmake.org/cmake/help/consulting.html > >> CMake Training Courses: http://cmake.org/cmake/help/training.html > >> > >> Visit other Kitware open-source projects at > >> http://www.kitware.com/opensource/opensource.html > >> > >> Follow this link to subscribe/unsubscribe: > >> http://public.kitware.com/mailman/listinfo/cmake > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davispuh at gmail.com Sat Aug 13 20:18:24 2016 From: davispuh at gmail.com (=?UTF-8?q?D=C4=81vis=20Mos=C4=81ns?=) Date: Sun, 14 Aug 2016 03:18:24 +0300 Subject: [cmake-developers] [PATCH v6] For Windows encode process output to internally used encoding In-Reply-To: <20160721172258.4366-1-davispuh@gmail.com> References: <20160721172258.4366-1-davispuh@gmail.com> Message-ID: <20160814001824.7760-1-davispuh@gmail.com> Typically Windows applications (eg. MSVC compiler) use current console's codepage for output to pipes so we need to encode that to internally used encoding (KWSYS_ENCODING_DEFAULT_CODEPAGE). --- Source/CMakeLists.txt | 2 + Source/ProcessOutput.hxx.in | 160 +++++++++++++++++++++++++++++++++++++ Source/cmExecProgramCommand.cxx | 3 + Source/cmExecuteProcessCommand.cxx | 11 ++- Source/cmProcessTools.cxx | 9 ++- Source/cmSystemTools.cxx | 11 ++- bootstrap | 3 + 7 files changed, 193 insertions(+), 6 deletions(-) create mode 100644 Source/ProcessOutput.hxx.in diff --git a/Source/CMakeLists.txt b/Source/CMakeLists.txt index a790994..eb51683 100644 --- a/Source/CMakeLists.txt +++ b/Source/CMakeLists.txt @@ -419,6 +419,8 @@ foreach(command_file endforeach() configure_file(cmCommands.cxx.in ${CMAKE_CURRENT_BINARY_DIR}/cmCommands.cxx @ONLY) +configure_file(ProcessOutput.hxx.in ${CMAKE_CURRENT_BINARY_DIR}/ProcessOutput.hxx) + # Kdevelop only works on UNIX and not windows if(UNIX) set(SRCS ${SRCS} cmGlobalKdevelopGenerator.cxx) diff --git a/Source/ProcessOutput.hxx.in b/Source/ProcessOutput.hxx.in new file mode 100644 index 0000000..ea7b881 --- /dev/null +++ b/Source/ProcessOutput.hxx.in @@ -0,0 +1,160 @@ +/*============================================================================ + CMake - Cross Platform Makefile Generator + Copyright 2000-2016 Kitware, Inc., Insight Software Consortium + + Distributed under the OSI-approved BSD License (the "License"); + see accompanying file Copyright.txt for details. + + This software is distributed WITHOUT ANY WARRANTY; without even the + implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + See the License for more information. +============================================================================*/ +#ifndef ProcessOutput_hxx +#define ProcessOutput_hxx + +#include +#include +#if defined(_WIN32) +# ifndef WIN32_LEAN_AND_MEAN +# define WIN32_LEAN_AND_MEAN 1 +# endif +# include +#endif + + class ProcessOutput + { + public: +#if defined(_WIN32) + static const UINT defaultCodepage = @KWSYS_ENCODING_DEFAULT_CODEPAGE@; +#endif + // must match to KWSYSPE_PIPE_BUFFER_SIZE + ProcessOutput(unsigned int maxSize = 1024) + { +#if defined(_WIN32) + bufferSize = maxSize; + codepage = GetConsoleCP(); + if (!codepage) { + codepage = GetACP(); + } +#else + static_cast(maxSize); +#endif + } + + ~ProcessOutput() + { + } + + bool DecodeText(std::string raw, std::string& decoded, size_t id = 0) + { + bool success = true; + decoded = raw; +#if defined(_WIN32) + if (id > 0) { + if (rawparts.size() < id) { + rawparts.reserve(id); + while (rawparts.size() < id) rawparts.push_back(std::string()); + } + raw = rawparts[id - 1] + raw; + rawparts[id - 1].clear(); + decoded = raw; + } + if (raw.size() > 0 && codepage != defaultCodepage) { + success = false; + CPINFOEXW cpinfo; + if (id > 0 && raw.size() == bufferSize && GetCPInfoExW(codepage, 0, &cpinfo) == 1 && cpinfo.MaxCharSize > 1) { + if (cpinfo.MaxCharSize == 2 && cpinfo.LeadByte[0] != 0) { + LPSTR prevChar = CharPrevExA(codepage, raw.c_str(), raw.c_str() + raw.size(), 0); + bool isLeadByte = (*(prevChar + 1) == 0) && IsDBCSLeadByteEx(codepage, *prevChar); + if (isLeadByte) { + rawparts[id - 1] += *(raw.end() - 1); + raw.resize(raw.size() - 1); + } + success = DoDecodeText(raw, decoded, NULL); + } else { + bool restoreDecoded = false; + std::string firstDecoded = decoded; + wchar_t lastChar = 0; + for (UINT i = 0; i < cpinfo.MaxCharSize; i++) { + success = DoDecodeText(raw, decoded, &lastChar); + if (success && lastChar != 0) { + if (i == 0) { + firstDecoded = decoded; + } + if (lastChar == cpinfo.UnicodeDefaultChar) { + restoreDecoded = true; + rawparts[id - 1] = *(raw.end() - 1) + rawparts[id - 1]; + raw.resize(raw.size() - 1); + } else { + restoreDecoded = false; + break; + } + } else { + break; + } + } + if (restoreDecoded) { + decoded = firstDecoded; + rawparts[id - 1].clear(); + } + } + } else { + success = DoDecodeText(raw, decoded, NULL); + } + } +#else + static_cast(id); +#endif + return success; + } + + bool DecodeText(const char* data, size_t length, std::string& decoded, size_t id = 0) + { + return DecodeText(std::string(data, length), decoded, id); + } + + bool DecodeText(std::vector raw, std::vector& decoded, size_t id = 0) + { + std::string str; + const bool success = DecodeText(std::string(raw.begin(), raw.end()), str, id); + decoded.assign(str.begin(), str.end()); + return success; + } + + private: +#if defined(_WIN32) + UINT codepage; + unsigned int bufferSize; + std::vector rawparts; + + bool DoDecodeText(std::string raw, std::string& decoded, wchar_t *lastChar) + { + bool success = false; + const int wlength = MultiByteToWideChar(codepage, 0, raw.c_str(), int(raw.size()), NULL, 0); + wchar_t* wdata = new wchar_t[wlength]; + int r = MultiByteToWideChar(codepage, 0, raw.c_str(), int(raw.size()), wdata, wlength); + if (r > 0) { + if (lastChar) { + *lastChar = 0; + if ((wlength >= 2 && wdata[wlength - 2] != wdata[wlength - 1]) || wlength >= 1) { + *lastChar = wdata[wlength - 1]; + } + } + int length = WideCharToMultiByte(defaultCodepage, 0, wdata, wlength, NULL, 0, NULL, NULL); + char *data = new char[length + 1]; + r = WideCharToMultiByte(defaultCodepage, 0, wdata, wlength, data, length, NULL, NULL); + if (r > 0) { + data[length] = '\0'; + decoded = data; + success = true; + } + delete[] data; + } + delete[] wdata; + return success; + } +#endif + }; + +#endif + diff --git a/Source/cmExecProgramCommand.cxx b/Source/cmExecProgramCommand.cxx index 58bbc31..fb2e11c 100644 --- a/Source/cmExecProgramCommand.cxx +++ b/Source/cmExecProgramCommand.cxx @@ -12,6 +12,7 @@ #include "cmExecProgramCommand.h" #include "cmSystemTools.h" +#include "ProcessOutput.hxx" #include @@ -219,6 +220,7 @@ bool cmExecProgramCommand::RunCommand(const char* command, std::string& output, int length; char* data; int p; + ProcessOutput processOutput; while ((p = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR), p)) { if (p == cmsysProcess_Pipe_STDOUT || p == cmsysProcess_Pipe_STDERR) { if (verbose) { @@ -230,6 +232,7 @@ bool cmExecProgramCommand::RunCommand(const char* command, std::string& output, // All output has been read. Wait for the process to exit. cmsysProcess_WaitForExit(cp, CM_NULLPTR); + processOutput.DecodeText(output, output); // Check the result of running the process. std::string msg; diff --git a/Source/cmExecuteProcessCommand.cxx b/Source/cmExecuteProcessCommand.cxx index d97b25f..38f1a15 100644 --- a/Source/cmExecuteProcessCommand.cxx +++ b/Source/cmExecuteProcessCommand.cxx @@ -12,6 +12,7 @@ #include "cmExecuteProcessCommand.h" #include "cmSystemTools.h" +#include "ProcessOutput.hxx" #include @@ -228,17 +229,21 @@ bool cmExecuteProcessCommand::InitialPass(std::vector const& args, int length; char* data; int p; + ProcessOutput processOutput; + std::string strdata; while ((p = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR), p)) { // Put the output in the right place. if (p == cmsysProcess_Pipe_STDOUT && !output_quiet) { if (output_variable.empty()) { - cmSystemTools::Stdout(data, length); + processOutput.DecodeText(data, length, strdata, 1); + cmSystemTools::Stdout(strdata.c_str(), strdata.size()); } else { cmExecuteProcessCommandAppend(tempOutput, data, length); } } else if (p == cmsysProcess_Pipe_STDERR && !error_quiet) { if (error_variable.empty()) { - cmSystemTools::Stderr(data, length); + processOutput.DecodeText(data, length, strdata, 2); + cmSystemTools::Stderr(strdata.c_str(), strdata.size()); } else { cmExecuteProcessCommandAppend(tempError, data, length); } @@ -247,6 +252,8 @@ bool cmExecuteProcessCommand::InitialPass(std::vector const& args, // All output has been read. Wait for the process to exit. cmsysProcess_WaitForExit(cp, CM_NULLPTR); + processOutput.DecodeText(tempOutput, tempOutput); + processOutput.DecodeText(tempError, tempError); // Fix the text in the output strings. cmExecuteProcessCommandFixText(tempOutput, output_strip_trailing_whitespace); diff --git a/Source/cmProcessTools.cxx b/Source/cmProcessTools.cxx index 34b8df2..a20a04c 100644 --- a/Source/cmProcessTools.cxx +++ b/Source/cmProcessTools.cxx @@ -10,6 +10,7 @@ See the License for more information. ============================================================================*/ #include "cmProcessTools.h" +#include "ProcessOutput.hxx" #include @@ -20,14 +21,18 @@ void cmProcessTools::RunProcess(struct cmsysProcess_s* cp, OutputParser* out, char* data = CM_NULLPTR; int length = 0; int p; + ProcessOutput processOutput; + std::string strdata; while ((out || err) && (p = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR), p)) { if (out && p == cmsysProcess_Pipe_STDOUT) { - if (!out->Process(data, length)) { + processOutput.DecodeText(data, length, strdata, 1); + if (!out->Process(strdata.c_str(), int(strdata.size()))) { out = CM_NULLPTR; } } else if (err && p == cmsysProcess_Pipe_STDERR) { - if (!err->Process(data, length)) { + processOutput.DecodeText(data, length, strdata, 2); + if (!err->Process(strdata.c_str(), int(strdata.size()))) { err = CM_NULLPTR; } } diff --git a/Source/cmSystemTools.cxx b/Source/cmSystemTools.cxx index 5745a01..da34d38 100644 --- a/Source/cmSystemTools.cxx +++ b/Source/cmSystemTools.cxx @@ -26,6 +26,7 @@ #include #include #include +#include "ProcessOutput.hxx" #if defined(CMAKE_BUILD_WITH_CMAKE) #include "cmArchiveWrite.h" #include "cmLocale.h" @@ -612,6 +613,8 @@ bool cmSystemTools::RunSingleCommand(std::vector const& command, char* data; int length; int pipe; + ProcessOutput processOutput; + std::string strdata; if (outputflag != OUTPUT_PASSTHROUGH && (captureStdOut || captureStdErr || outputflag != OUTPUT_NONE)) { while ((pipe = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR)) > @@ -627,14 +630,16 @@ bool cmSystemTools::RunSingleCommand(std::vector const& command, if (pipe == cmsysProcess_Pipe_STDOUT) { if (outputflag != OUTPUT_NONE) { - cmSystemTools::Stdout(data, length); + processOutput.DecodeText(data, length, strdata, 1); + cmSystemTools::Stdout(strdata.c_str(), strdata.size()); } if (captureStdOut) { tempStdOut.insert(tempStdOut.end(), data, data + length); } } else if (pipe == cmsysProcess_Pipe_STDERR) { if (outputflag != OUTPUT_NONE) { - cmSystemTools::Stderr(data, length); + processOutput.DecodeText(data, length, strdata, 2); + cmSystemTools::Stderr(strdata.c_str(), strdata.size()); } if (captureStdErr) { tempStdErr.insert(tempStdErr.end(), data, data + length); @@ -646,9 +651,11 @@ bool cmSystemTools::RunSingleCommand(std::vector const& command, cmsysProcess_WaitForExit(cp, CM_NULLPTR); if (captureStdOut) { captureStdOut->assign(tempStdOut.begin(), tempStdOut.end()); + processOutput.DecodeText(*captureStdOut, *captureStdOut); } if (captureStdErr) { captureStdErr->assign(tempStdErr.begin(), tempStdErr.end()); + processOutput.DecodeText(*captureStdErr, *captureStdErr); } bool result = true; diff --git a/bootstrap b/bootstrap index 742fa2b..ae36e54 100755 --- a/bootstrap +++ b/bootstrap @@ -1294,6 +1294,9 @@ for h in Configure VersionConfig; do fi done +cmake_replace_string "${cmake_source_dir}/Source/ProcessOutput.hxx.in" \ + "${cmake_bootstrap_dir}/ProcessOutput.hxx" KWSYS_ENCODING_DEFAULT_CODEPAGE CP_ACP + # Prepare KWSYS cmake_kwsys_config_replace_string \ "${cmake_source_dir}/Source/kwsys/Configure.hxx.in" \ -- 2.9.2 From insertinterestingnamehere at gmail.com Sat Aug 13 20:32:39 2016 From: insertinterestingnamehere at gmail.com (Ian Henriksen) Date: Sun, 14 Aug 2016 00:32:39 +0000 Subject: [cmake-developers] [CMake] Need ideas/opinions on third party library management In-Reply-To: References: Message-ID: On Sat, Aug 13, 2016 at 6:43 PM Elizabeth A. Fischer < elizabeth.fischer at columbia.edu> wrote: > I would look into Anaconda, which does work for Windows. Its version > management is not as well developed as Spack, but it's more cross-platform. > > Auto-builders are just coming into their own, it's a brave new world. I > expect things to be more complete in a few years. > > -- Elizabeth > > Yes, Anaconda comes with a package manager (conda) that is designed to manage binary dependencies. You can get a minimal Python install with just the package manager by installing miniconda. The package management system is built in Python, but it can manage C and C++ libraries as well. The support for ARM isn't thoroughly built out, so you'll be building all the packages there for yourself. The support for Windwos, OS X and Linux is pretty well developed. You can look at https://repo.continuum.io/pkgs/ to see the packages available by default. There is also support for people uploading their own "channels" for packages on anaconda.org. The conda-forge channel in particular has a lot of additional packages. Hopefully that's more like what you're looking for. Best, Ian Henriksen -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.lamaison at gmail.com Sun Aug 14 13:58:52 2016 From: alexander.lamaison at gmail.com (Alexander Lamaison) Date: Sun, 14 Aug 2016 18:58:52 +0100 Subject: [cmake-developers] FindOpenSSL: Link libraries needed for static OpenSSL. Message-ID: OpenSSL depends on system libraries. When linking statically again OpenSSL, the client target must also link those libraries. This patch updates FindOpenSSL.cmake to include the necessary system libs in the OpenSSL libraries variables and in the OpenSSL::Crypto and OpenSSL::SSL target properties. The dependencies: Windows: crypt32 Unix: CMAKE_DL_LIBS -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-FindOpenSSL-Link-libraries-needed-for-static-OpenSSL.patch Type: text/x-patch Size: 3975 bytes Desc: not available URL: From seblist at xwmw.org Mon Aug 15 03:23:54 2016 From: seblist at xwmw.org (Sebastian Holtermann) Date: Mon, 15 Aug 2016 09:23:54 +0200 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> <6298f722-911e-d9ba-8998-3355ad481482@xwmw.org> Message-ID: Am 10.08.2016 um 19:33 schrieb Brad King: > On 08/10/2016 06:15 AM, Sebastian Holtermann wrote: >> These patches change the private abstract interface of cmCryptoHash to >> return std::vector instead of std::string. >> Conversion to std::string is handled by a static public method. > > Nice. I've added this to the topic, now merged for testing here: > > Merge topic 'autogen-same-name' into next > https://cmake.org/gitweb?p=cmake.git;a=commit;h=df237068 Thanks! > Now all this topic is missing is an update to the documentation > to explain the behavior w.r.t. duplicate file names versus the > symbol names. > Like this? -Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-AUTORCC-documentation-extended-for-same-.qrc-name-ca.patch Type: text/x-patch Size: 1005 bytes Desc: not available URL: From brad.king at kitware.com Mon Aug 15 09:25:44 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 15 Aug 2016 09:25:44 -0400 Subject: [cmake-developers] [PATCH v6] For Windows encode process output to internally used encoding In-Reply-To: <20160814001824.7760-1-davispuh@gmail.com> References: <20160721172258.4366-1-davispuh@gmail.com> <20160814001824.7760-1-davispuh@gmail.com> Message-ID: On 08/13/2016 08:18 PM, D?vis Mos?ns wrote: > Source/ProcessOutput.hxx.in | 160 +++++++++++++++++++++++++++++++++++++ [snip] > + static const UINT defaultCodepage = @KWSYS_ENCODING_DEFAULT_CODEPAGE@; Thanks. Overall the approach looks good now. I'd like to avoid having to configure this header just for the default codepage value. Also I'd like to avoid having to include `windows.h` publicly. Please move the implementation into a .cxx file to hide these parts away from includers. Thanks, -Brad From brad.king at kitware.com Mon Aug 15 09:25:53 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 15 Aug 2016 09:25:53 -0400 Subject: [cmake-developers] FindOpenSSL: Link libraries needed for static OpenSSL. In-Reply-To: References: Message-ID: On 08/14/2016 01:58 PM, Alexander Lamaison wrote: > This patch updates FindOpenSSL.cmake to include the necessary system > libs in the OpenSSL libraries variables and in the OpenSSL::Crypto and > OpenSSL::SSL target properties. Thanks. The singluar-named OPENSSL_{SSL,CRYPTO}_LIBRARY variables are meant just to hold the specific library files (i.e. the result of one find_library call). Instead of placing the dependencies directly in them we should have other (internal) variables to hold them. That will also avoid duplicating conditions for filling in the imported target information. -Brad From brad.king at kitware.com Mon Aug 15 09:26:00 2016 From: brad.king at kitware.com (Brad King) Date: Mon, 15 Aug 2016 09:26:00 -0400 Subject: [cmake-developers] QtAutogen 3.6.0 directory nesting overflow patch In-Reply-To: References: <695ab7bc-0e4d-aa9b-3198-7e77056830d3@xwmw.org> <7c7ff822-1fb4-91f0-b79b-8ba6b950f3ed@kitware.com> <6e42456b-a181-4b4c-5ec7-2a7eb941b738@xwmw.org> <8461c374-02dc-72dc-cb37-5394b12e58e4@kitware.com> <5da3b473-22e3-91e9-6fb4-7d19e3b82e0e@xwmw.org> <1b37a61a-f7ae-bf87-2849-443cc10928a8@kitware.com> <6298f722-911e-d9ba-8998-3355ad481482@xwmw.org> Message-ID: <9528fd91-482f-d058-bbc4-f8f85d0239de@kitware.com> On 08/15/2016 03:23 AM, Sebastian Holtermann wrote: > Like this? Thanks, applied. I've merged the series to `master` now: Merge topic 'cmCryptoHash-raw-digest' https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=f203694b Merge topic 'autogen-same-name' https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=33d4aff5 Thanks, -Brad From tobias.hunger at gmail.com Mon Aug 15 09:56:14 2016 From: tobias.hunger at gmail.com (Tobias Hunger) Date: Mon, 15 Aug 2016 15:56:14 +0200 Subject: [cmake-developers] cmake -E capabilities [v3] Message-ID: Hi CMake developers, I am back to a internet-connected world, so here is a new update of the cmake -E capabilities patch. I did the changes Brad requested: * Rebased on cmake master branch * Add documentation * Add a smoke test that makes sure that "cmake -E capabilities" can run. Code is on github again: https://github.com/hunger/CMake/commits/cmake-capabilities Best Regards, Tobias From alex at rogue-research.com Mon Aug 15 14:24:46 2016 From: alex at rogue-research.com (Alex Ciobanu) Date: Mon, 15 Aug 2016 19:24:46 +0100 Subject: [cmake-developers] llvm coverage format .profdata Message-ID: <3D3B96AB-F158-4C2C-9424-3EB59C85E380@rogue-research.com> Hi, As many of you know, clang now outputs coverage information using the LLVM format described here: http://llvm.org/docs/CoverageMappingFormat.html Notably, Xcode 7.3 and later uses this format for its code coverage functionality. Currently CMake/CTest looks for the following types of files: Cannot find any GCov coverage files. Not a valid Intel Coverage command. Cannot find any Python Trace.py coverage files. Cannot find Cobertura XML file: .../coverage.xml Cannot find GTM coverage file: .../gtm_coverage.mcov Cannot find Cache coverage file: .../cache_coverage.cmcov Cannot find Jacoco coverage files: .../*jacoco.xml Cannot find BlanketJS coverage files: .../*.json Cannot find any coverage files. Ignoring Coverage request. Cannot find Delphi coverage files: .../*(*.pas).html The .profdata format for LLVM coverage data does not seem to be supported yet. Are there any plans in this direction? Also is anyone aware of a way to get this coverage data displayed on the CDash dashboard, maybe by converting it to another format? sincerely, Alex Ciobanu -------------- next part -------------- An HTML attachment was scrubbed... URL: From davispuh at gmail.com Mon Aug 15 16:34:21 2016 From: davispuh at gmail.com (=?UTF-8?q?D=C4=81vis=20Mos=C4=81ns?=) Date: Mon, 15 Aug 2016 23:34:21 +0300 Subject: [cmake-developers] [PATCH v7] For Windows encode process output to internally used encoding Message-ID: <20160815203421.28064-1-davispuh@gmail.com> Typically Windows applications (eg. MSVC compiler) use current console's codepage for output to pipes so we need to encode that to internally used encoding (KWSYS_ENCODING_DEFAULT_CODEPAGE). --- Source/CMakeLists.txt | 6 ++ Source/ProcessOutput.cxx | 141 +++++++++++++++++++++++++++++++++++++ Source/ProcessOutput.hxx | 39 ++++++++++ Source/cmExecProgramCommand.cxx | 3 + Source/cmExecuteProcessCommand.cxx | 11 ++- Source/cmProcessTools.cxx | 9 ++- Source/cmSystemTools.cxx | 11 ++- bootstrap | 5 +- 8 files changed, 218 insertions(+), 7 deletions(-) create mode 100644 Source/ProcessOutput.cxx create mode 100644 Source/ProcessOutput.hxx diff --git a/Source/CMakeLists.txt b/Source/CMakeLists.txt index cdc8fb1..46dd471 100644 --- a/Source/CMakeLists.txt +++ b/Source/CMakeLists.txt @@ -373,8 +373,14 @@ set(SRCS cm_sha2.c cm_utf8.h cm_utf8.c + + ProcessOutput.cxx + ProcessOutput.hxx ) +SET_PROPERTY(SOURCE ProcessOutput.cxx APPEND PROPERTY COMPILE_DEFINITIONS + KWSYS_ENCODING_DEFAULT_CODEPAGE=${KWSYS_ENCODING_DEFAULT_CODEPAGE}) + set(COMMAND_INCLUDES "#include \"cmTargetPropCommandBase.cxx\"\n") list(APPEND SRCS cmTargetPropCommandBase.cxx) set_property(SOURCE cmTargetPropCommandBase.cxx PROPERTY HEADER_FILE_ONLY ON) diff --git a/Source/ProcessOutput.cxx b/Source/ProcessOutput.cxx new file mode 100644 index 0000000..6c66087 --- /dev/null +++ b/Source/ProcessOutput.cxx @@ -0,0 +1,141 @@ +/*============================================================================ + CMake - Cross Platform Makefile Generator + Copyright 2000-2016 Kitware, Inc., Insight Software Consortium + + Distributed under the OSI-approved BSD License (the "License"); + see accompanying file Copyright.txt for details. + + This software is distributed WITHOUT ANY WARRANTY; without even the + implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + See the License for more information. +============================================================================*/ + +#include "ProcessOutput.hxx" + +#if defined(_WIN32) +# include +unsigned int ProcessOutput::defaultCodepage = KWSYS_ENCODING_DEFAULT_CODEPAGE; +#endif + +ProcessOutput::ProcessOutput(unsigned int maxSize) +{ +#if defined(_WIN32) + bufferSize = maxSize; + codepage = GetConsoleCP(); + if (!codepage) { + codepage = GetACP(); + } +#else + static_cast(maxSize); +#endif +} + +ProcessOutput::~ProcessOutput() +{ +} + +bool ProcessOutput::DecodeText(std::string raw, std::string& decoded, size_t id) +{ + bool success = true; + decoded = raw; +#if defined(_WIN32) + if (id > 0) { + if (rawparts.size() < id) { + rawparts.reserve(id); + while (rawparts.size() < id) rawparts.push_back(std::string()); + } + raw = rawparts[id - 1] + raw; + rawparts[id - 1].clear(); + decoded = raw; + } + if (raw.size() > 0 && codepage != defaultCodepage) { + success = false; + CPINFOEXW cpinfo; + if (id > 0 && raw.size() == bufferSize && GetCPInfoExW(codepage, 0, &cpinfo) == 1 && cpinfo.MaxCharSize > 1) { + if (cpinfo.MaxCharSize == 2 && cpinfo.LeadByte[0] != 0) { + LPSTR prevChar = CharPrevExA(codepage, raw.c_str(), raw.c_str() + raw.size(), 0); + bool isLeadByte = (*(prevChar + 1) == 0) && IsDBCSLeadByteEx(codepage, *prevChar); + if (isLeadByte) { + rawparts[id - 1] += *(raw.end() - 1); + raw.resize(raw.size() - 1); + } + success = DoDecodeText(raw, decoded, NULL); + } else { + bool restoreDecoded = false; + std::string firstDecoded = decoded; + wchar_t lastChar = 0; + for (UINT i = 0; i < cpinfo.MaxCharSize; i++) { + success = DoDecodeText(raw, decoded, &lastChar); + if (success && lastChar != 0) { + if (i == 0) { + firstDecoded = decoded; + } + if (lastChar == cpinfo.UnicodeDefaultChar) { + restoreDecoded = true; + rawparts[id - 1] = *(raw.end() - 1) + rawparts[id - 1]; + raw.resize(raw.size() - 1); + } else { + restoreDecoded = false; + break; + } + } else { + break; + } + } + if (restoreDecoded) { + decoded = firstDecoded; + rawparts[id - 1].clear(); + } + } + } else { + success = DoDecodeText(raw, decoded, NULL); + } + } +#else + static_cast(id); +#endif + return success; +} + +bool ProcessOutput::DecodeText(const char* data, size_t length, std::string& decoded, size_t id) +{ + return DecodeText(std::string(data, length), decoded, id); +} + +bool ProcessOutput::DecodeText(std::vector raw, std::vector& decoded, size_t id) +{ + std::string str; + const bool success = DecodeText(std::string(raw.begin(), raw.end()), str, id); + decoded.assign(str.begin(), str.end()); + return success; +} + +#if defined(_WIN32) +bool ProcessOutput::DoDecodeText(std::string raw, std::string& decoded, wchar_t *lastChar) +{ + bool success = false; + const int wlength = MultiByteToWideChar(codepage, 0, raw.c_str(), int(raw.size()), NULL, 0); + wchar_t* wdata = new wchar_t[wlength]; + int r = MultiByteToWideChar(codepage, 0, raw.c_str(), int(raw.size()), wdata, wlength); + if (r > 0) { + if (lastChar) { + *lastChar = 0; + if ((wlength >= 2 && wdata[wlength - 2] != wdata[wlength - 1]) || wlength >= 1) { + *lastChar = wdata[wlength - 1]; + } + } + int length = WideCharToMultiByte(defaultCodepage, 0, wdata, wlength, NULL, 0, NULL, NULL); + char *data = new char[length + 1]; + r = WideCharToMultiByte(defaultCodepage, 0, wdata, wlength, data, length, NULL, NULL); + if (r > 0) { + data[length] = '\0'; + decoded = data; + success = true; + } + delete[] data; + } + delete[] wdata; + return success; +} +#endif + diff --git a/Source/ProcessOutput.hxx b/Source/ProcessOutput.hxx new file mode 100644 index 0000000..a988a01 --- /dev/null +++ b/Source/ProcessOutput.hxx @@ -0,0 +1,39 @@ +/*============================================================================ + CMake - Cross Platform Makefile Generator + Copyright 2000-2016 Kitware, Inc., Insight Software Consortium + + Distributed under the OSI-approved BSD License (the "License"); + see accompanying file Copyright.txt for details. + + This software is distributed WITHOUT ANY WARRANTY; without even the + implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. + See the License for more information. +============================================================================*/ +#ifndef ProcessOutput_hxx +#define ProcessOutput_hxx + +#include +#include + +class ProcessOutput +{ + public: + static unsigned int defaultCodepage; + // must match to KWSYSPE_PIPE_BUFFER_SIZE + ProcessOutput(unsigned int maxSize = 1024); + ~ProcessOutput(); + bool DecodeText(std::string raw, std::string& decoded, size_t id = 0); + bool DecodeText(const char* data, size_t length, std::string& decoded, size_t id = 0); + bool DecodeText(std::vector raw, std::vector& decoded, size_t id = 0); + + private: + unsigned int codepage; + unsigned int bufferSize; + std::vector rawparts; +#if defined(_WIN32) + bool DoDecodeText(std::string raw, std::string& decoded, wchar_t *lastChar); +#endif +}; + +#endif + diff --git a/Source/cmExecProgramCommand.cxx b/Source/cmExecProgramCommand.cxx index 58bbc31..fb2e11c 100644 --- a/Source/cmExecProgramCommand.cxx +++ b/Source/cmExecProgramCommand.cxx @@ -12,6 +12,7 @@ #include "cmExecProgramCommand.h" #include "cmSystemTools.h" +#include "ProcessOutput.hxx" #include @@ -219,6 +220,7 @@ bool cmExecProgramCommand::RunCommand(const char* command, std::string& output, int length; char* data; int p; + ProcessOutput processOutput; while ((p = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR), p)) { if (p == cmsysProcess_Pipe_STDOUT || p == cmsysProcess_Pipe_STDERR) { if (verbose) { @@ -230,6 +232,7 @@ bool cmExecProgramCommand::RunCommand(const char* command, std::string& output, // All output has been read. Wait for the process to exit. cmsysProcess_WaitForExit(cp, CM_NULLPTR); + processOutput.DecodeText(output, output); // Check the result of running the process. std::string msg; diff --git a/Source/cmExecuteProcessCommand.cxx b/Source/cmExecuteProcessCommand.cxx index d97b25f..38f1a15 100644 --- a/Source/cmExecuteProcessCommand.cxx +++ b/Source/cmExecuteProcessCommand.cxx @@ -12,6 +12,7 @@ #include "cmExecuteProcessCommand.h" #include "cmSystemTools.h" +#include "ProcessOutput.hxx" #include @@ -228,17 +229,21 @@ bool cmExecuteProcessCommand::InitialPass(std::vector const& args, int length; char* data; int p; + ProcessOutput processOutput; + std::string strdata; while ((p = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR), p)) { // Put the output in the right place. if (p == cmsysProcess_Pipe_STDOUT && !output_quiet) { if (output_variable.empty()) { - cmSystemTools::Stdout(data, length); + processOutput.DecodeText(data, length, strdata, 1); + cmSystemTools::Stdout(strdata.c_str(), strdata.size()); } else { cmExecuteProcessCommandAppend(tempOutput, data, length); } } else if (p == cmsysProcess_Pipe_STDERR && !error_quiet) { if (error_variable.empty()) { - cmSystemTools::Stderr(data, length); + processOutput.DecodeText(data, length, strdata, 2); + cmSystemTools::Stderr(strdata.c_str(), strdata.size()); } else { cmExecuteProcessCommandAppend(tempError, data, length); } @@ -247,6 +252,8 @@ bool cmExecuteProcessCommand::InitialPass(std::vector const& args, // All output has been read. Wait for the process to exit. cmsysProcess_WaitForExit(cp, CM_NULLPTR); + processOutput.DecodeText(tempOutput, tempOutput); + processOutput.DecodeText(tempError, tempError); // Fix the text in the output strings. cmExecuteProcessCommandFixText(tempOutput, output_strip_trailing_whitespace); diff --git a/Source/cmProcessTools.cxx b/Source/cmProcessTools.cxx index 34b8df2..a20a04c 100644 --- a/Source/cmProcessTools.cxx +++ b/Source/cmProcessTools.cxx @@ -10,6 +10,7 @@ See the License for more information. ============================================================================*/ #include "cmProcessTools.h" +#include "ProcessOutput.hxx" #include @@ -20,14 +21,18 @@ void cmProcessTools::RunProcess(struct cmsysProcess_s* cp, OutputParser* out, char* data = CM_NULLPTR; int length = 0; int p; + ProcessOutput processOutput; + std::string strdata; while ((out || err) && (p = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR), p)) { if (out && p == cmsysProcess_Pipe_STDOUT) { - if (!out->Process(data, length)) { + processOutput.DecodeText(data, length, strdata, 1); + if (!out->Process(strdata.c_str(), int(strdata.size()))) { out = CM_NULLPTR; } } else if (err && p == cmsysProcess_Pipe_STDERR) { - if (!err->Process(data, length)) { + processOutput.DecodeText(data, length, strdata, 2); + if (!err->Process(strdata.c_str(), int(strdata.size()))) { err = CM_NULLPTR; } } diff --git a/Source/cmSystemTools.cxx b/Source/cmSystemTools.cxx index 5745a01..da34d38 100644 --- a/Source/cmSystemTools.cxx +++ b/Source/cmSystemTools.cxx @@ -26,6 +26,7 @@ #include #include #include +#include "ProcessOutput.hxx" #if defined(CMAKE_BUILD_WITH_CMAKE) #include "cmArchiveWrite.h" #include "cmLocale.h" @@ -612,6 +613,8 @@ bool cmSystemTools::RunSingleCommand(std::vector const& command, char* data; int length; int pipe; + ProcessOutput processOutput; + std::string strdata; if (outputflag != OUTPUT_PASSTHROUGH && (captureStdOut || captureStdErr || outputflag != OUTPUT_NONE)) { while ((pipe = cmsysProcess_WaitForData(cp, &data, &length, CM_NULLPTR)) > @@ -627,14 +630,16 @@ bool cmSystemTools::RunSingleCommand(std::vector const& command, if (pipe == cmsysProcess_Pipe_STDOUT) { if (outputflag != OUTPUT_NONE) { - cmSystemTools::Stdout(data, length); + processOutput.DecodeText(data, length, strdata, 1); + cmSystemTools::Stdout(strdata.c_str(), strdata.size()); } if (captureStdOut) { tempStdOut.insert(tempStdOut.end(), data, data + length); } } else if (pipe == cmsysProcess_Pipe_STDERR) { if (outputflag != OUTPUT_NONE) { - cmSystemTools::Stderr(data, length); + processOutput.DecodeText(data, length, strdata, 2); + cmSystemTools::Stderr(strdata.c_str(), strdata.size()); } if (captureStdErr) { tempStdErr.insert(tempStdErr.end(), data, data + length); @@ -646,9 +651,11 @@ bool cmSystemTools::RunSingleCommand(std::vector const& command, cmsysProcess_WaitForExit(cp, CM_NULLPTR); if (captureStdOut) { captureStdOut->assign(tempStdOut.begin(), tempStdOut.end()); + processOutput.DecodeText(*captureStdOut, *captureStdOut); } if (captureStdErr) { captureStdErr->assign(tempStdErr.begin(), tempStdErr.end()); + processOutput.DecodeText(*captureStdErr, *captureStdErr); } bool result = true; diff --git a/bootstrap b/bootstrap index 742fa2b..ba4ddb5 100755 --- a/bootstrap +++ b/bootstrap @@ -328,6 +328,7 @@ CMAKE_CXX_SOURCES="\ cmExprLexer \ cmExprParser \ cmExprParserHelper \ + ProcessOutput \ " if ${cmake_system_mingw}; then @@ -1343,6 +1344,7 @@ fi cmake_c_flags_String="-DKWSYS_STRING_C" if ${cmake_system_mingw}; then cmake_c_flags_EncodingC="-DKWSYS_ENCODING_DEFAULT_CODEPAGE=CP_ACP" + cmake_cxx_flags_ProcessOutput=${cmake_c_flags_EncodingC} fi cmake_cxx_flags_SystemTools=" -DKWSYS_CXX_HAS_SETENV=${KWSYS_CXX_HAS_SETENV} @@ -1359,8 +1361,9 @@ echo "cmake: ${objs}" > "${cmake_bootstrap_dir}/Makefile" echo " ${cmake_cxx_compiler} ${cmake_ld_flags} ${cmake_cxx_flags} ${objs} -o cmake" >> "${cmake_bootstrap_dir}/Makefile" for a in ${CMAKE_CXX_SOURCES}; do src=`cmake_escape "${cmake_source_dir}/Source/${a}.cxx"` + src_flags=`eval echo \\${cmake_cxx_flags_\${a}}` echo "${a}.o : ${src} ${dep}" >> "${cmake_bootstrap_dir}/Makefile" - echo " ${cmake_cxx_compiler} ${cmake_cxx_flags} -c ${src} -o ${a}.o" >> "${cmake_bootstrap_dir}/Makefile" + echo " ${cmake_cxx_compiler} ${cmake_cxx_flags} ${src_flags} -c ${src} -o ${a}.o" >> "${cmake_bootstrap_dir}/Makefile" done echo "cmBootstrapCommands1.o : $cmBootstrapCommands1Deps" >> "${cmake_bootstrap_dir}/Makefile" echo "cmBootstrapCommands2.o : $cmBootstrapCommands2Deps" >> "${cmake_bootstrap_dir}/Makefile" -- 2.9.3 From michael.stuermer at schaeffler.com Tue Aug 16 02:13:42 2016 From: michael.stuermer at schaeffler.com (Stuermer, Michael SP/HZA-ZSEP) Date: Tue, 16 Aug 2016 06:13:42 +0000 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> References: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> Message-ID: I'll provide a patch where I change it to patching the actual feature XML tags. Michael > -----Original Message----- > From: Nils Gladitz [mailto:nilsgladitz at gmail.com] > Sent: Friday, August 12, 2016 1:50 PM > To: Stuermer, Michael SP/HZA-ZSEP; CMake Developers > Subject: Re: [cmake-developers] [Patch 5/5] Improved WIX support > > On 08/12/2016 11:50 AM, Stuermer, Michael SP/HZA-ZSEP wrote: > > > > >> Patch 5 seems to implement patching of FeatureRef rather than the > >> original Feature elements. > >> I am not sure if this is what you intended but this could be error > >> prone given that there could in theory be any number (0-n) FeatureRef > >> elements for any corresponding Feature element. > >> > >> Nils > > The intention was to be able to add custom components that are added as > extra .wxs source files to certain features. If there are more convenient ways > to do this I will be happy to change the implementation or adapt my WIX > project. But so far this seemed to be a very easy and intuitive solution: the > additional component references are added in the same place where all > other component references are added as well. > > I understand the general intention but not why you elected to patch > FeatureRef elements instead of the Feature elements themselves. > > This would not be any more convenient but would certainly match > expectations and be less ill defined. > e.g. when I specify a patch for a Feature I expect that the Feature with the > given ID gets patched. > > Arguments against patching a FeatureRef instead are: > - There can be n FeatureRef elements for any Feature element in which case > it would not be obvious if the patch should be applied to one > (which?) or all of these > - While similar FeatureRef and Feature don't take the same Child elements > - You can already define your own FeatureRef elements (referencing any of > the pre-existing Feature elements if wanted) without having to use the > patch mechanism > > Nils > . From charles.huet at gmail.com Tue Aug 16 03:28:04 2016 From: charles.huet at gmail.com (Charles Huet) Date: Tue, 16 Aug 2016 07:28:04 +0000 Subject: [cmake-developers] [CMake] Need ideas/opinions on third party library management In-Reply-To: References: Message-ID: Hi, I was looking at tools that can do this kind of things myself (however I was more looking at pre-built binaries redistribution than at a super-build, since our build time is already quite long). Does Conan (https://conan.io/) not fit your bill as well ? Best Le dim. 14 ao?t 2016 ? 02:33, Ian Henriksen < insertinterestingnamehere at gmail.com> a ?crit : > On Sat, Aug 13, 2016 at 6:43 PM Elizabeth A. Fischer < > elizabeth.fischer at columbia.edu> wrote: > >> I would look into Anaconda, which does work for Windows. Its version >> management is not as well developed as Spack, but it's more cross-platform. >> >> Auto-builders are just coming into their own, it's a brave new world. I >> expect things to be more complete in a few years. >> >> -- Elizabeth >> >> > Yes, Anaconda comes with a package manager (conda) that is designed to > manage binary dependencies. You can get a minimal Python install with just > the > package manager by installing miniconda. The package management system is > built in Python, but it can manage C and C++ libraries as well. The > support for > ARM isn't thoroughly built out, so you'll be building all the packages > there for > yourself. The support for Windwos, OS X and Linux is pretty well > developed. You > can look at https://repo.continuum.io/pkgs/ to see the packages available > by > default. There is also support for people uploading their own "channels" > for > packages on anaconda.org. The conda-forge channel in particular has a lot > of > additional packages. > Hopefully that's more like what you're looking for. > Best, > Ian Henriksen > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.stuermer at schaeffler.com Tue Aug 16 04:15:01 2016 From: michael.stuermer at schaeffler.com (Stuermer, Michael SP/HZA-ZSEP) Date: Tue, 16 Aug 2016 08:15:01 +0000 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> References: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> Message-ID: <7d2346e2cd9e44ed8698e81f93e90390@DE013666.schaeffler.com> > -----Original Message----- > From: Nils Gladitz [mailto:nilsgladitz at gmail.com] > Sent: Friday, August 12, 2016 1:50 PM > To: Stuermer, Michael SP/HZA-ZSEP; CMake Developers > Subject: Re: [cmake-developers] [Patch 5/5] Improved WIX support > > On 08/12/2016 11:50 AM, Stuermer, Michael SP/HZA-ZSEP wrote: > > > > >> Patch 5 seems to implement patching of FeatureRef rather than the > >> original Feature elements. > >> I am not sure if this is what you intended but this could be error > >> prone given that there could in theory be any number (0-n) FeatureRef > >> elements for any corresponding Feature element. > >> > >> Nils > > The intention was to be able to add custom components that are added as > extra .wxs source files to certain features. If there are more convenient ways > to do this I will be happy to change the implementation or adapt my WIX > project. But so far this seemed to be a very easy and intuitive solution: the > additional component references are added in the same place where all > other component references are added as well. > > I understand the general intention but not why you elected to patch > FeatureRef elements instead of the Feature elements themselves. > After having look at the code for some minutes I remember why patching the ref instead oft he feature was my way to go: The feature is written to file in cmWIXFeaturesSourceWriter::EmitFeatureForComponent(), where I do not have any patch information available. This means I'd have to change the signature of both EmitFeatureForComponent and EmitFeatureForComponentGroup and pass a reference to the patch instance along. Multiple occurrence of IDs can happen, but the patch will only be applied once because applied fragments are erased immediately after writing them to the stream. So after all for me this was a consideration of a 1-line change vs. changing class interfaces an passing object instances to where it might not be desirable. I agree the commit message of the patch is not accurate enough and if there is way to add custom WIX-components to features without changing the cpack source I'd be happy to do so. But so far I tried several approaches and neither worked (see below). > This would not be any more convenient but would certainly match > expectations and be less ill defined. > e.g. when I specify a patch for a Feature I expect that the Feature with the > given ID gets patched. > > Arguments against patching a FeatureRef instead are: > - There can be n FeatureRef elements for any Feature element in which case > it would not be obvious if the patch should be applied to one > (which?) or all of these The way the patch was implemented only the featurerefs in the generated features.wxs file would be patched and there should not be any double occurences of a feature ref. > - While similar FeatureRef and Feature don't take the same Child elements Right, and if both Feature and FeatureRef would be patchable we would be in trouble. For the lazy one: this is not the case at the moment so we would not need to worry about it (but it's very nice). For the correct one: We could introduce another attribute to CPackWixFragment called "Type" where type of the XML node to be patched could be stored. But this would introduce additional complexity to the cmWIXPatch class... > - You can already define your own FeatureRef elements (referencing any of > the pre-existing Feature elements if wanted) without having to use the > patch mechanism > I tried this like this (in a separate additional .wxs source file added with CPACK_WIX_EXTRA_SOURCES): Did not work, the registry value was not set. Using the proposed approach it worked. Do I have to reference it somehow different? > Nils > . Michael From nilsgladitz at gmail.com Tue Aug 16 04:54:27 2016 From: nilsgladitz at gmail.com (Nils Gladitz) Date: Tue, 16 Aug 2016 10:54:27 +0200 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: <7d2346e2cd9e44ed8698e81f93e90390@DE013666.schaeffler.com> References: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> <7d2346e2cd9e44ed8698e81f93e90390@DE013666.schaeffler.com> Message-ID: <18345fa8-15c5-eed7-1353-b8b2f9ccf5ab@gmail.com> On 08/16/2016 10:15 AM, Stuermer, Michael SP/HZA-ZSEP wrote: > > After having look at the code for some minutes I remember why patching the ref instead oft he feature was my way to go: > > The feature is written to file in cmWIXFeaturesSourceWriter::EmitFeatureForComponent(), where I do not have any patch information available. This means I'd have to change the signature of both EmitFeatureForComponent and EmitFeatureForComponentGroup and pass a reference to the patch instance along. Multiple occurrence of IDs can happen, but the patch will only be applied once because applied fragments are erased immediately after writing them to the stream. > > So after all for me this was a consideration of a 1-line change vs. changing class interfaces an passing object instances to where it might not be desirable. There is precedence in cmWIXFilesSourceWriter::EmitComponentFile() so I think such an interface change would be fine. > > I agree the commit message of the patch is not accurate enough and if there is way to add custom WIX-components to features without changing the cpack source I'd be happy to do so. But so far I tried several approaches and neither worked (see below). > >> This would not be any more convenient but would certainly match >> expectations and be less ill defined. >> e.g. when I specify a patch for a Feature I expect that the Feature with the >> given ID gets patched. >> >> Arguments against patching a FeatureRef instead are: >> - There can be n FeatureRef elements for any Feature element in which case >> it would not be obvious if the patch should be applied to one >> (which?) or all of these > The way the patch was implemented only the featurerefs in the generated features.wxs file would be patched and there should not be any double occurences of a feature ref. > >> - While similar FeatureRef and Feature don't take the same Child elements > Right, and if both Feature and FeatureRef would be patchable we would be in trouble. For the lazy one: this is not the case at the moment so we would not need to worry about it (but it's very nice). For the correct one: We could introduce another attribute to CPackWixFragment called "Type" where type of the XML node to be patched could be stored. But this would introduce additional complexity to the cmWIXPatch class... There is no use case to be able to patch both FeatureRef and Feature elements when Feature elements can be patched directly. > >> - You can already define your own FeatureRef elements (referencing any of >> the pre-existing Feature elements if wanted) without having to use the >> patch mechanism >> > I tried this like this (in a separate additional .wxs source file added with CPACK_WIX_EXTRA_SOURCES): > > > > > > > > > > > > > > > Did not work, the registry value was not set. Using the proposed approach it worked. Do I have to reference it somehow different? The linker only includes object files which provide a symbol that is required by an object already included. Your source file has a single symbol for the Component "SetRegistryValues" but that symbol (I assume) is not required by any of the other objects which the linker includes. You could e.g. add the FeatureRef to a custom WIX.template.in (which has the main entry point and is therefor always included), or supply a patch for the Product element (#PRODUCT), or create any kind of valid reference to your custom source file (if any reference is resolved through your custom source the entire object gets included). Nils From michael.stuermer at schaeffler.com Tue Aug 16 05:15:31 2016 From: michael.stuermer at schaeffler.com (Stuermer, Michael SP/HZA-ZSEP) Date: Tue, 16 Aug 2016 09:15:31 +0000 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: <18345fa8-15c5-eed7-1353-b8b2f9ccf5ab@gmail.com> References: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> <7d2346e2cd9e44ed8698e81f93e90390@DE013666.schaeffler.com> <18345fa8-15c5-eed7-1353-b8b2f9ccf5ab@gmail.com> Message-ID: <8b081282e1d743a6a7c0a0723417e399@DE013666.schaeffler.com> > -----Original Message----- > From: Nils Gladitz [mailto:nilsgladitz at gmail.com] > Sent: Tuesday, August 16, 2016 10:54 AM > To: Stuermer, Michael SP/HZA-ZSEP; CMake Developers > Subject: Re: [cmake-developers] [Patch 5/5] Improved WIX support > > On 08/16/2016 10:15 AM, Stuermer, Michael SP/HZA-ZSEP wrote: > > > > > After having look at the code for some minutes I remember why patching > the ref instead oft he feature was my way to go: > > > > The feature is written to file in > cmWIXFeaturesSourceWriter::EmitFeatureForComponent(), where I do not > have any patch information available. This means I'd have to change the > signature of both EmitFeatureForComponent and > EmitFeatureForComponentGroup and pass a reference to the patch instance > along. Multiple occurrence of IDs can happen, but the patch will only be > applied once because applied fragments are erased immediately after > writing them to the stream. > > > > So after all for me this was a consideration of a 1-line change vs. changing > class interfaces an passing object instances to where it might not be > desirable. > > There is precedence in cmWIXFilesSourceWriter::EmitComponentFile() so I > think such an interface change would be fine. > Ok I'll do this. Should solve all issues and doubts hopefully. > > > > I agree the commit message of the patch is not accurate enough and if > there is way to add custom WIX-components to features without changing > the cpack source I'd be happy to do so. But so far I tried several approaches > and neither worked (see below). > > > >> This would not be any more convenient but would certainly match > >> expectations and be less ill defined. > >> e.g. when I specify a patch for a Feature I expect that the Feature > >> with the given ID gets patched. > >> > >> Arguments against patching a FeatureRef instead are: > >> - There can be n FeatureRef elements for any Feature element in which > >> case it would not be obvious if the patch should be applied to one > >> (which?) or all of these > > The way the patch was implemented only the featurerefs in the generated > features.wxs file would be patched and there should not be any double > occurences of a feature ref. > > > >> - While similar FeatureRef and Feature don't take the same Child > >> elements > > Right, and if both Feature and FeatureRef would be patchable we would be > in trouble. For the lazy one: this is not the case at the moment so we would > not need to worry about it (but it's very nice). For the correct one: We could ... meant NOT very nice, of course ... > introduce another attribute to CPackWixFragment called "Type" where type > of the XML node to be patched could be stored. But this would introduce > additional complexity to the cmWIXPatch class... > > There is no use case to be able to patch both FeatureRef and Feature > elements when Feature elements can be patched directly. > Right. > > > >> - You can already define your own FeatureRef elements (referencing > >> any of the pre-existing Feature elements if wanted) without having to > >> use the patch mechanism > >> > > I tried this like this (in a separate additional .wxs source file added with > CPACK_WIX_EXTRA_SOURCES): > > > > > > > > Directory="INSTALL_ROOT"> > > > > > > > > > > IgnoreParent="yes"> > > > > > > > > > > > > Did not work, the registry value was not set. Using the proposed approach > it worked. Do I have to reference it somehow different? > > The linker only includes object files which provide a symbol that is required > by an object already included. > Your source file has a single symbol for the Component "SetRegistryValues" > but that symbol (I assume) is not required by any of the other objects which > the linker includes. > > You could e.g. add the FeatureRef to a custom WIX.template.in (which has > the main entry point and is therefor always included), or supply a patch for > the Product element (#PRODUCT), or create any kind of valid reference to > your custom source file (if any reference is resolved through your custom > source the entire object gets included). > Adding FeatureRef to #PRODUCT does not work. I get the following message: features.wixobj : error LGHT0095 : Multiple primary references were found for Feature '@feature@' in Feature 'ProductFeature' and Product '{@guid@}' > Nils > . Michael From nilsgladitz at gmail.com Tue Aug 16 06:51:38 2016 From: nilsgladitz at gmail.com (Nils Gladitz) Date: Tue, 16 Aug 2016 12:51:38 +0200 Subject: [cmake-developers] [Patch 5/5] Improved WIX support In-Reply-To: <8b081282e1d743a6a7c0a0723417e399@DE013666.schaeffler.com> References: <95f264e5-ea18-ac48-6974-86b8ce167e95@gmail.com> <7d2346e2cd9e44ed8698e81f93e90390@DE013666.schaeffler.com> <18345fa8-15c5-eed7-1353-b8b2f9ccf5ab@gmail.com> <8b081282e1d743a6a7c0a0723417e399@DE013666.schaeffler.com> Message-ID: <81f17829-dcea-fa00-ac39-3942d4fc3384@gmail.com> On 08/16/2016 11:15 AM, Stuermer, Michael SP/HZA-ZSEP wrote: >> There is precedence in cmWIXFilesSourceWriter::EmitComponentFile() so I >> think such an interface change would be fine. >> > Ok I'll do this. Should solve all issues and doubts hopefully. Great. Thanks. > > Adding FeatureRef to #PRODUCT does not work. I get the following message: > > features.wixobj : error LGHT0095 : Multiple primary references were found for Feature '@feature@' in Feature 'ProductFeature' and Product '{@guid@}' Did you try with IgnoreParent="yes" on your FeatureRef element? It makes sense that there can only be one root-Feature. Nils From florent.castelli at gmail.com Tue Aug 16 06:52:50 2016 From: florent.castelli at gmail.com (Florent Castelli) Date: Tue, 16 Aug 2016 12:52:50 +0200 Subject: [cmake-developers] Need ideas/opinions on third party library management In-Reply-To: References: Message-ID: <1D3F7CC1-58CE-4B01-A2E2-9AFBB746B384@gmail.com> At Spotify, we use CMake a lot for our large C++ library shared by all the clients. After trying to build libraries for each platform and variant, we basically gave up and we now use a super-build approach. For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android and iOS. Each platform has a different CPU target (or many 32/64bit, x86/ARM). Each platform has many compilers. Some platforms have instrumentation options (Debug / Release, ASan, MSan?) and really need to be compiled properly, otherwise you?ll end up with false positives. The matrix of builds is REALLY hard to track. Each time we update Boost, we had to update a lot of things. I tried using ExternalProject and use b2 (build tool from Boost) to build it and instead of having lots of build jobs with a mirror of the flags, you end up mirroring the flags in your CMake files instead, which is still not good enough. In the end, I looked at how Boost is actually built. And for most libraries, it?s plain simple. A static library with a few files, some define, sometimes a platform specific source file. What if instead of having an external build tool, I built it from CMake instead? It would propagate all the build flags, target, instrumentation and compiler information from the main build to it and just work. I tried it and it worked in no time! We replaced our Boost 1.59 binary distribution with the source distribution and it?s much easier. When people build our library for a different target, they don?t have to download new binaries, they just reuse the same sources. Later on, we found a bug in Boost 1.59 (fixed in later versions) and patched it. We updated our source bundle and everything was smooth. Much later on, we wanted to use 1.61. We just updated the source bundle again, the list of source files or compilation flags for the libraries we use didn?t change. It was again effortless. Overall, building boost takes 10s on our developers? machines. The sources aren?t changed often, so the cost is pretty low. It needs attention when we upgrade it, but that?s quite rare. We try now to use the same approach for other libraries when we add them. Some of them are already using CMake and it?s somewhat easier, but since most people still target version 2.8 (or 2.6...), we find it better to rewrite the build scripts ourselves and use modern features (as in, everything is a target that propagates requirements, we don?t propagate variables). It makes it also much easier to build a library for another platform that wasn?t targeted by the original project. If people are interested, I could share the CMakeLists.txt file we use for Boost. It doesn?t build all the libraries (some are hard like Context) and uses some internal macros, but it should be plain simple to tweak for your use. /Florent > On 12 Aug 2016, at 21:59, Robert Dailey wrote: > > Hello, > > There is an internal C++ product at the company I work for which I > have written a series of CMake scripts for. This project actually has > dependencies on several open source libraries, such as boost, > freetype, openssl, etc. > > Right now what we do is build each of these third party libraries *by > hand*, once for every platform we support (Windows, Linux x86, Android > NDK). Then we stuff the includes (headers) and libraries > (static/shared) in a submodule and the primary code base's CMake > scripts pull them in as interface targets. > > This works well and is light-weight but is a pain when upgrading or > changing libraries. It's a pain because if I want to upgrade boost, I > have to build it up to 6 times (once for each platform and once for > each configuration). > > I've been thinking of a different approach for a while. I've done some > toying around with the "Super Build" concept, where I have a separate > CMake project that does nothing but use the ExternalProject module to > build libraries in real time along with our project. So the order of > operations would be as follows (for our automated build server): > > 1. Clone our "Third Party" repository > 2. Use CMake to generate & build the "Super Build" project (this > builds boost, openssl, freetype, etc for the current platform). > 3. Clone the main code base's repository > 4. Use CMake to generate & build, using find_package() to refer to > interface targets exported by those third party libraries built in > step 2 > > Obviously this will make builds take significantly longer, because > we're constantly rebuilding the same third party libraries over and > over again. However, it virtually eliminates the maintenance burden > for third party libraries because they are built inherently with > everything else. > > Note that I can't refer to pre-built libraries in our build > environments because we need very specific control over the versions > of our libraries as well as the toolchains that were used to build > them. Also we may specifically build our libraries a certain way (such > as boost). For this reason we do not rely on our external environment > or external package managers to fulfill third party dependencies, like > most open source projects do on Linux for example. > > Does this "Super Build" approach sound like a better idea? What other > options are available? The downside with the "Super Build" solution is > that it will become very difficult to make the transition between > building third party and building our code base seamless. I can't do > both in the same generate step because find_package() can't be called > until the libraries are built & installed. > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers From kernelmd at yandex.ru Tue Aug 16 07:11:48 2016 From: kernelmd at yandex.ru (Titov Denis) Date: Tue, 16 Aug 2016 14:11:48 +0300 Subject: [cmake-developers] [PATCH] cmFileCommand: Download continuation support Message-ID: <211201471345908@web22j.yandex.ru> This patch introduces several options for the file(DOWNLOAD ...) command, which would be useful in case of dealing with big files or unstable network connections. The added options are: RETRY_COUNT -- sets maximal amount of download restarts, default value: 1 RETRY_DELAY -- sets delay before restarting download (in seconds), default value: 0.0 RETRY_MAX_TIME -- sets maximal time spent in downloading file (in seconds), default value: infinity RETRY_CONTINUE -- if set, makes cmake try to continue downloading of the existing chunk, instead of discarding it and starting all over. This option is not set by default Notes: The RETRY_CONTINUE option requires server-side support of http partial get (content-range header). Unfortunately, I haven't been able to properly test the RETRY_CONTINUE option, as I didn't have access to the appropriate server. Any help in this area is encouraged. --- Source/cmFileCommand.cxx | 287 +++++++++++++++++++++++++++++++++-------------- 1 file changed, 204 insertions(+), 83 deletions(-) diff --git a/Source/cmFileCommand.cxx b/Source/cmFileCommand.cxx index 835b118..aa7782e 100644 --- a/Source/cmFileCommand.cxx +++ b/Source/cmFileCommand.cxx @@ -34,6 +34,16 @@ // include sys/stat.h after sys/types.h #include +#include +#include + +// For crossplatform_sleep(). +#if defined(_WIN32) && !defined(__CYGWIN__) +#include +#else +#include +#endif + #include #include #include @@ -68,6 +78,15 @@ static mode_t mode_setuid = S_ISUID; static mode_t mode_setgid = S_ISGID; #endif +void crossplatform_sleep(int delay_s) +{ +#if defined(_WIN32) && !defined(__CYGWIN__) + Sleep(delay_s); +#else + sleep(delay_s * 1000); +#endif +} + #if defined(_WIN32) && defined(CMAKE_ENCODING_UTF8) // libcurl doesn't support file:// urls for unicode filenames on Windows. // Convert string from UTF-8 to ACP if this is a file:// URL. @@ -2481,6 +2500,11 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) std::string hashMatchMSG; CM_AUTO_PTR hash; bool showProgress = false; + int retryMaxCount = 1; + double retryDelayS = 0.0; + double retryMaxTimeS = DBL_MAX; + bool retryContinue = false; + cmsys::ofstream fout; while (i != args.end()) { if (*i == "TIMEOUT") { @@ -2564,7 +2588,34 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) return false; } hashMatchMSG = algo + " hash"; + } else if (*i == "RETRY_COUNT") { + ++i; + if (i != args.end()) { + retryMaxCount = atoi(i->c_str()); + } else { + this->SetError("DOWNLOAD missing count for RETRY_COUNT"); + return false; + } + } else if (*i == "RETRY_DELAY") { + ++i; + if (i != args.end()) { + retryDelayS = atof(i->c_str()); + } else { + this->SetError("DOWNLOAD missing time for RETRY_DELAY"); + return false; + } + } else if (*i == "RETRY_MAX_TIME") { + ++i; + if (i != args.end()) { + retryMaxTimeS = atof(i->c_str()); + } else { + this->SetError("DOWNLOAD missing time for RETRY_MAX_TIME"); + return false; + } + } else if (*i == "RETRY_CONTINUE") { + retryContinue = true; } + ++i; } // If file exists already, and caller specified an expected md5 or sha, @@ -2599,110 +2650,171 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) return false; } - cmsys::ofstream fout(file.c_str(), std::ios::binary); - if (!fout) { - this->SetError("DOWNLOAD cannot open file for write."); - return false; - } - #if defined(_WIN32) && defined(CMAKE_ENCODING_UTF8) url = fix_file_url_windows(url); #endif + cmFileCommandVectorOfChar chunkDebug; + ::CURL* curl; ::curl_global_init(CURL_GLOBAL_DEFAULT); - curl = ::curl_easy_init(); - if (!curl) { - this->SetError("DOWNLOAD error initializing curl."); - return false; - } - cURLEasyGuard g_curl(curl); - ::CURLcode res = ::curl_easy_setopt(curl, CURLOPT_URL, url.c_str()); - check_curl_result(res, "DOWNLOAD cannot set url: "); + ::CURLcode res; + int tries = 0; + double elapsed = 0.0; + time_t start, end; + while (tries < retryMaxCount && elapsed <= retryMaxTimeS) { + ++tries; + time(&start); + + curl = ::curl_easy_init(); + if (!curl) { + this->SetError("DOWNLOAD error initializing curl."); + ::curl_global_cleanup(); + return false; + } - // enable HTTP ERROR parsing - res = ::curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1); - check_curl_result(res, "DOWNLOAD cannot set http failure option: "); + if (cmSystemTools::FileExists(file.c_str())) { // Something was downloaded. + // Check hash. + if (hash.get()) { + std::string actualHash = hash->HashFile(file); + if (actualHash == expectedHash) { // File is complete, exit. + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return true; + } + } - res = ::curl_easy_setopt(curl, CURLOPT_USERAGENT, "curl/" LIBCURL_VERSION); - check_curl_result(res, "DOWNLOAD cannot set user agent option: "); + if (retryContinue == false) { // Discard downloaded chunk. + fout.open(file.c_str(), std::ios::binary | std::ios::trunc); + if (!fout.good()) { + this->SetError("Cannot open file for writing"); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } + } else { // Try to continue. + fout.open(file.c_str(), std::ios::binary | std::ios::app); + if (!fout.good()) { + this->SetError("Cannot open file for writing"); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } + curl_easy_setopt(curl, CURLOPT_RESUME_FROM_LARGE, fout.tellp()); + } + } else { // Create new file. + fout.open(file.c_str(), std::ios::binary); + if (!fout.good()) { + this->SetError("Cannot open file for writing"); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } + } - res = ::curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, cmWriteToFileCallback); - check_curl_result(res, "DOWNLOAD cannot set write function: "); + cURLEasyGuard g_curl(curl); + res = ::curl_easy_setopt(curl, CURLOPT_URL, url.c_str()); + check_curl_result(res, "DOWNLOAD cannot set url: "); - res = ::curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, - cmFileCommandCurlDebugCallback); - check_curl_result(res, "DOWNLOAD cannot set debug function: "); + // enable HTTP ERROR parsing + res = ::curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1); + check_curl_result(res, "DOWNLOAD cannot set http failure option: "); - // check to see if TLS verification is requested - if (tls_verify) { - res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1); - check_curl_result(res, "Unable to set TLS/SSL Verify on: "); - } else { - res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0); - check_curl_result(res, "Unable to set TLS/SSL Verify off: "); - } - // check to see if a CAINFO file has been specified - // command arg comes first - std::string const& cainfo_err = cmCurlSetCAInfo(curl, cainfo); - if (!cainfo_err.empty()) { - this->SetError(cainfo_err); - return false; - } + res = ::curl_easy_setopt(curl, CURLOPT_USERAGENT, "curl/" LIBCURL_VERSION); + check_curl_result(res, "DOWNLOAD cannot set user agent option: "); - cmFileCommandVectorOfChar chunkDebug; + res = + ::curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, cmWriteToFileCallback); + check_curl_result(res, "DOWNLOAD cannot set write function: "); - res = ::curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void*)&fout); - check_curl_result(res, "DOWNLOAD cannot set write data: "); + res = ::curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, + cmFileCommandCurlDebugCallback); + check_curl_result(res, "DOWNLOAD cannot set debug function: "); - res = ::curl_easy_setopt(curl, CURLOPT_DEBUGDATA, (void*)&chunkDebug); - check_curl_result(res, "DOWNLOAD cannot set debug data: "); + // check to see if TLS verification is requested + if (tls_verify) { + res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1); + check_curl_result(res, "Unable to set TLS/SSL Verify on: "); + } else { + res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0); + check_curl_result(res, "Unable to set TLS/SSL Verify off: "); + } + // check to see if a CAINFO file has been specified + // command arg comes first + std::string const& cainfo_err = cmCurlSetCAInfo(curl, cainfo); + if (!cainfo_err.empty()) { + this->SetError(cainfo_err); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } - res = ::curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); - check_curl_result(res, "DOWNLOAD cannot set follow-redirect option: "); + res = ::curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void*)&fout); + check_curl_result(res, "DOWNLOAD cannot set write data: "); - if (!logVar.empty()) { - res = ::curl_easy_setopt(curl, CURLOPT_VERBOSE, 1); - check_curl_result(res, "DOWNLOAD cannot set verbose: "); - } + res = ::curl_easy_setopt(curl, CURLOPT_DEBUGDATA, (void*)&chunkDebug); + check_curl_result(res, "DOWNLOAD cannot set debug data: "); - if (timeout > 0) { - res = ::curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout); - check_curl_result(res, "DOWNLOAD cannot set timeout: "); - } + res = ::curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); + check_curl_result(res, "DOWNLOAD cannot set follow-redirect option: "); - if (inactivity_timeout > 0) { - // Give up if there is no progress for a long time. - ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1); - ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, inactivity_timeout); - } + if (!logVar.empty()) { + res = ::curl_easy_setopt(curl, CURLOPT_VERBOSE, 1); + check_curl_result(res, "DOWNLOAD cannot set verbose: "); + } - // Need the progress helper's scope to last through the duration of - // the curl_easy_perform call... so this object is declared at function - // scope intentionally, rather than inside the "if(showProgress)" - // block... - // - cURLProgressHelper helper(this, "download"); + if (timeout > 0) { + res = ::curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout); + check_curl_result(res, "DOWNLOAD cannot set timeout: "); + } - if (showProgress) { - res = ::curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0); - check_curl_result(res, "DOWNLOAD cannot set noprogress value: "); + if (inactivity_timeout > 0) { + // Give up if there is no progress for a long time. + ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1); + ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, inactivity_timeout); + } - res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, - cmFileDownloadProgressCallback); - check_curl_result(res, "DOWNLOAD cannot set progress function: "); + // Need the progress helper's scope to last through the duration of + // the curl_easy_perform call... so this object is declared at loop + // scope intentionally, rather than inside the "if(showProgress)" + // block... + // + cURLProgressHelper helper(this, "download"); - res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, - reinterpret_cast(&helper)); - check_curl_result(res, "DOWNLOAD cannot set progress data: "); - } + if (showProgress) { + res = ::curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0); + check_curl_result(res, "DOWNLOAD cannot set noprogress value: "); - res = ::curl_easy_perform(curl); + res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, + cmFileDownloadProgressCallback); + check_curl_result(res, "DOWNLOAD cannot set progress function: "); - /* always cleanup */ - g_curl.release(); - ::curl_easy_cleanup(curl); + res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, + reinterpret_cast(&helper)); + check_curl_result(res, "DOWNLOAD cannot set progress data: "); + } + + res = ::curl_easy_perform(curl); + + /* always cleanup */ + g_curl.release(); + ::curl_easy_cleanup(curl); + fout.flush(); + fout.close(); + + // Download finished successfuly, exit the loop. + if (res == ::CURLE_OK) { + break; + // Server doesn't support content ranges... + } else if (retryContinue == true && res == ::CURLE_RANGE_ERROR) { + retryContinue = false; // Disable download continuation. + } + + crossplatform_sleep(retryDelayS); + time(&end); + elapsed += difftime(end, start); + } if (!statusVar.empty()) { std::ostringstream result; @@ -2712,10 +2824,19 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) ::curl_global_cleanup(); - // Explicitly flush/close so we can measure the md5 accurately. - // - fout.flush(); - fout.close(); + if (res != ::CURLE_OK) { + std::ostringstream oss; + // Failed by exhausting attempts + if (retryMaxCount != 1 && tries == retryMaxCount) { + oss << "Download failed after " << tries << " attempts. "; + } + // Failed by exhausting maximal time. + if (retryMaxTimeS < DBL_MAX && elapsed >= retryMaxTimeS) { + oss << "Download failed: time exhausted, " << elapsed << "s. spent. "; + } + oss << "Last CURL error: " << curl_easy_strerror(res); + return false; + } // Verify MD5 sum if requested: // -- 2.9.2 From bballet at ivsweb.com Tue Aug 16 08:41:29 2016 From: bballet at ivsweb.com (Benjamin Ballet) Date: Tue, 16 Aug 2016 14:41:29 +0200 Subject: [cmake-developers] Need ideas/opinions on third party library management In-Reply-To: <1D3F7CC1-58CE-4B01-A2E2-9AFBB746B384@gmail.com> References: <1D3F7CC1-58CE-4B01-A2E2-9AFBB746B384@gmail.com> Message-ID: Very interesting discussion, we have the same issues here. Florent Castelli, how many third parties libraries do you use ? I think a super build can be a very good solution but I'm wondering how much third party code you have to build. Here we use OpenCV, with, boost, and poco, and other things... So it may be too long. I was personnaly considering having an hybrid solution : include small libraries (like jsoncpp) and pre-build the other for each platforms. 2016-08-16 12:52 GMT+02:00 Florent Castelli : > At Spotify, we use CMake a lot for our large C++ library shared by all the > clients. > After trying to build libraries for each platform and variant, we > basically gave up and we now > use a super-build approach. > > For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android > and iOS. > Each platform has a different CPU target (or many 32/64bit, x86/ARM). > Each platform has many compilers. > Some platforms have instrumentation options (Debug / Release, ASan, MSan?) > and really need > to be compiled properly, otherwise you?ll end up with false positives. > The matrix of builds is REALLY hard to track. Each time we update Boost, > we had to update > a lot of things. > I tried using ExternalProject and use b2 (build tool from Boost) to build > it and instead of having > lots of build jobs with a mirror of the flags, you end up mirroring the > flags in your CMake files > instead, which is still not good enough. > > In the end, I looked at how Boost is actually built. And for most > libraries, it?s plain simple. > A static library with a few files, some define, sometimes a platform > specific source file. > What if instead of having an external build tool, I built it from CMake > instead? > It would propagate all the build flags, target, instrumentation and > compiler information from the main > build to it and just work. > I tried it and it worked in no time! We replaced our Boost 1.59 binary > distribution with the source > distribution and it?s much easier. When people build our library for a > different target, they don?t have > to download new binaries, they just reuse the same sources. > Later on, we found a bug in Boost 1.59 (fixed in later versions) and > patched it. We updated our source > bundle and everything was smooth. > Much later on, we wanted to use 1.61. We just updated the source bundle > again, the list of source > files or compilation flags for the libraries we use didn?t change. It was > again effortless. > > Overall, building boost takes 10s on our developers? machines. The sources > aren?t changed often, > so the cost is pretty low. It needs attention when we upgrade it, but > that?s quite rare. > > We try now to use the same approach for other libraries when we add them. > Some of them are > already using CMake and it?s somewhat easier, but since most people still > target version 2.8 (or 2.6...), > we find it better to rewrite the build scripts ourselves and use modern > features (as in, everything is > a target that propagates requirements, we don?t propagate variables). > It makes it also much easier to build a library for another platform that > wasn?t targeted by the original > project. > > If people are interested, I could share the CMakeLists.txt file we use for > Boost. It doesn?t build all > the libraries (some are hard like Context) and uses some internal macros, > but it should be plain > simple to tweak for your use. > > /Florent > > > On 12 Aug 2016, at 21:59, Robert Dailey > wrote: > > > > Hello, > > > > There is an internal C++ product at the company I work for which I > > have written a series of CMake scripts for. This project actually has > > dependencies on several open source libraries, such as boost, > > freetype, openssl, etc. > > > > Right now what we do is build each of these third party libraries *by > > hand*, once for every platform we support (Windows, Linux x86, Android > > NDK). Then we stuff the includes (headers) and libraries > > (static/shared) in a submodule and the primary code base's CMake > > scripts pull them in as interface targets. > > > > This works well and is light-weight but is a pain when upgrading or > > changing libraries. It's a pain because if I want to upgrade boost, I > > have to build it up to 6 times (once for each platform and once for > > each configuration). > > > > I've been thinking of a different approach for a while. I've done some > > toying around with the "Super Build" concept, where I have a separate > > CMake project that does nothing but use the ExternalProject module to > > build libraries in real time along with our project. So the order of > > operations would be as follows (for our automated build server): > > > > 1. Clone our "Third Party" repository > > 2. Use CMake to generate & build the "Super Build" project (this > > builds boost, openssl, freetype, etc for the current platform). > > 3. Clone the main code base's repository > > 4. Use CMake to generate & build, using find_package() to refer to > > interface targets exported by those third party libraries built in > > step 2 > > > > Obviously this will make builds take significantly longer, because > > we're constantly rebuilding the same third party libraries over and > > over again. However, it virtually eliminates the maintenance burden > > for third party libraries because they are built inherently with > > everything else. > > > > Note that I can't refer to pre-built libraries in our build > > environments because we need very specific control over the versions > > of our libraries as well as the toolchains that were used to build > > them. Also we may specifically build our libraries a certain way (such > > as boost). For this reason we do not rely on our external environment > > or external package managers to fulfill third party dependencies, like > > most open source projects do on Linux for example. > > > > Does this "Super Build" approach sound like a better idea? What other > > options are available? The downside with the "Super Build" solution is > > that it will become very difficult to make the transition between > > building third party and building our code base seamless. I can't do > > both in the same generate step because find_package() can't be called > > until the libraries are built & installed. > > -- > > > > Powered by www.kitware.com > > > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > > > CMake Support: http://cmake.org/cmake/help/support.html > > CMake Consulting: http://cmake.org/cmake/help/consulting.html > > CMake Training Courses: http://cmake.org/cmake/help/training.html > > > > Visit other Kitware open-source projects at http://www.kitware.com/ > opensource/opensource.html > > > > Follow this link to subscribe/unsubscribe: > > http://public.kitware.com/mailman/listinfo/cmake-developers > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/ > opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > -- *Benjamin BALLET* Ing?nieur R&D *ACTIVISU* 19, rue Klock - 92110 Clichy *> Standard T?l* : 01 44 69 37 37 *>* www.activisu.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From elizabeth.fischer at columbia.edu Tue Aug 16 08:52:16 2016 From: elizabeth.fischer at columbia.edu (Elizabeth A. Fischer) Date: Tue, 16 Aug 2016 08:52:16 -0400 Subject: [cmake-developers] Need ideas/opinions on third party library management In-Reply-To: <1D3F7CC1-58CE-4B01-A2E2-9AFBB746B384@gmail.com> References: <1D3F7CC1-58CE-4B01-A2E2-9AFBB746B384@gmail.com> Message-ID: CMake builds for existing libraries are certainly an interesting and useful thing, and deserve to be posted in a GitHub repo somewhere. They should also serve as the basis of a campaign to get the library authors to incorporate the CMake build directly in their repos. But any approach that requires every build to be ported to CMake will be difficult and labor-prone to scale. Writing a meta-build recipe is usually much easier. Spack handles the combinatorial dependencies you mention in a sophisticated, graceful way that most meta-builders do not. Its only problem is it does not (yet) run on Windows. There's no fundamental reason why not; we just need someone to get involved and start trying it on Windows. -- Elizabeth On Tue, Aug 16, 2016 at 6:52 AM, Florent Castelli < florent.castelli at gmail.com> wrote: > At Spotify, we use CMake a lot for our large C++ library shared by all the > clients. > After trying to build libraries for each platform and variant, we > basically gave up and we now > use a super-build approach. > > For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android > and iOS. > Each platform has a different CPU target (or many 32/64bit, x86/ARM). > Each platform has many compilers. > Some platforms have instrumentation options (Debug / Release, ASan, MSan?) > and really need > to be compiled properly, otherwise you?ll end up with false positives. > The matrix of builds is REALLY hard to track. Each time we update Boost, > we had to update > a lot of things. > I tried using ExternalProject and use b2 (build tool from Boost) to build > it and instead of having > lots of build jobs with a mirror of the flags, you end up mirroring the > flags in your CMake files > instead, which is still not good enough. > > In the end, I looked at how Boost is actually built. And for most > libraries, it?s plain simple. > A static library with a few files, some define, sometimes a platform > specific source file. > What if instead of having an external build tool, I built it from CMake > instead? > It would propagate all the build flags, target, instrumentation and > compiler information from the main > build to it and just work. > I tried it and it worked in no time! We replaced our Boost 1.59 binary > distribution with the source > distribution and it?s much easier. When people build our library for a > different target, they don?t have > to download new binaries, they just reuse the same sources. > Later on, we found a bug in Boost 1.59 (fixed in later versions) and > patched it. We updated our source > bundle and everything was smooth. > Much later on, we wanted to use 1.61. We just updated the source bundle > again, the list of source > files or compilation flags for the libraries we use didn?t change. It was > again effortless. > > Overall, building boost takes 10s on our developers? machines. The sources > aren?t changed often, > so the cost is pretty low. It needs attention when we upgrade it, but > that?s quite rare. > > We try now to use the same approach for other libraries when we add them. > Some of them are > already using CMake and it?s somewhat easier, but since most people still > target version 2.8 (or 2.6...), > we find it better to rewrite the build scripts ourselves and use modern > features (as in, everything is > a target that propagates requirements, we don?t propagate variables). > It makes it also much easier to build a library for another platform that > wasn?t targeted by the original > project. > > If people are interested, I could share the CMakeLists.txt file we use for > Boost. It doesn?t build all > the libraries (some are hard like Context) and uses some internal macros, > but it should be plain > simple to tweak for your use. > > /Florent > > > On 12 Aug 2016, at 21:59, Robert Dailey > wrote: > > > > Hello, > > > > There is an internal C++ product at the company I work for which I > > have written a series of CMake scripts for. This project actually has > > dependencies on several open source libraries, such as boost, > > freetype, openssl, etc. > > > > Right now what we do is build each of these third party libraries *by > > hand*, once for every platform we support (Windows, Linux x86, Android > > NDK). Then we stuff the includes (headers) and libraries > > (static/shared) in a submodule and the primary code base's CMake > > scripts pull them in as interface targets. > > > > This works well and is light-weight but is a pain when upgrading or > > changing libraries. It's a pain because if I want to upgrade boost, I > > have to build it up to 6 times (once for each platform and once for > > each configuration). > > > > I've been thinking of a different approach for a while. I've done some > > toying around with the "Super Build" concept, where I have a separate > > CMake project that does nothing but use the ExternalProject module to > > build libraries in real time along with our project. So the order of > > operations would be as follows (for our automated build server): > > > > 1. Clone our "Third Party" repository > > 2. Use CMake to generate & build the "Super Build" project (this > > builds boost, openssl, freetype, etc for the current platform). > > 3. Clone the main code base's repository > > 4. Use CMake to generate & build, using find_package() to refer to > > interface targets exported by those third party libraries built in > > step 2 > > > > Obviously this will make builds take significantly longer, because > > we're constantly rebuilding the same third party libraries over and > > over again. However, it virtually eliminates the maintenance burden > > for third party libraries because they are built inherently with > > everything else. > > > > Note that I can't refer to pre-built libraries in our build > > environments because we need very specific control over the versions > > of our libraries as well as the toolchains that were used to build > > them. Also we may specifically build our libraries a certain way (such > > as boost). For this reason we do not rely on our external environment > > or external package managers to fulfill third party dependencies, like > > most open source projects do on Linux for example. > > > > Does this "Super Build" approach sound like a better idea? What other > > options are available? The downside with the "Super Build" solution is > > that it will become very difficult to make the transition between > > building third party and building our code base seamless. I can't do > > both in the same generate step because find_package() can't be called > > until the libraries are built & installed. > > -- > > > > Powered by www.kitware.com > > > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > > > CMake Support: http://cmake.org/cmake/help/support.html > > CMake Consulting: http://cmake.org/cmake/help/consulting.html > > CMake Training Courses: http://cmake.org/cmake/help/training.html > > > > Visit other Kitware open-source projects at http://www.kitware.com/ > opensource/opensource.html > > > > Follow this link to subscribe/unsubscribe: > > http://public.kitware.com/mailman/listinfo/cmake-developers > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/ > opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -------------- next part -------------- An HTML attachment was scrubbed... URL: From florent.castelli at gmail.com Tue Aug 16 09:03:15 2016 From: florent.castelli at gmail.com (Florent Castelli) Date: Tue, 16 Aug 2016 15:03:15 +0200 Subject: [cmake-developers] Need ideas/opinions on third party library management In-Reply-To: References: <1D3F7CC1-58CE-4B01-A2E2-9AFBB746B384@gmail.com> Message-ID: We have many smaller libraries and a few big ones like Boost. It certainly takes a bit of time to setup, but it will save you a lot of time later. It will keep your build scripts clean too as the user won?t have to know about its dependencies setup. Just use ?target_link_libraries? on the modern target that was created and it will work. Most of the 3rd party dependencies don?t work that way and they're quite painful to use correctly. As in, if you miss to add the ?add_definitions? they told you to use, it will build, but source files might be used with a different preprocessor define and you end up with two different implementations of the same class in the same binary. It will crash in very mysterious ways. The most important point is *correctness*. If you experiment with tooling like the Clang sanitizers, you?ll know that you can?t mix libraries that haven?t been instrumented with instrumented code. Best case the libraries are self-contained and will work all of the time. Worst case, the libraries are mostly self-contained and will *not* work sometimes! As an example, the Boost Unit Test Framework usually worked fine for us. But sometimes, we had ASan triggering an issue in the constructor of the string object for the test name. Our binary was built with ASan and Boost without. When we built it from source, all the false positives went away. I know that not a lot of people use them or have heard of them, but they are a super important tool that will find TONS of real issues in your code the first time you use them and we cannot do without. Some people might rely on Valgrind Memcheck instead, but hey, this is 2016 and we like our tests to run fast(er)! /Florent > On 16 Aug 2016, at 14:41, Benjamin Ballet wrote: > > Very interesting discussion, we have the same issues here. > > Florent Castelli, how many third parties libraries do you use ? I think a super build can be a very good solution but I'm wondering how much third party code you have to build. Here we use OpenCV, with, boost, and poco, and other things... So it may be too long. > > I was personnaly considering having an hybrid solution : include small libraries (like jsoncpp) and pre-build the other for each platforms. > > > 2016-08-16 12:52 GMT+02:00 Florent Castelli >: > At Spotify, we use CMake a lot for our large C++ library shared by all the clients. > After trying to build libraries for each platform and variant, we basically gave up and we now > use a super-build approach. > > For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android and iOS. > Each platform has a different CPU target (or many 32/64bit, x86/ARM). > Each platform has many compilers. > Some platforms have instrumentation options (Debug / Release, ASan, MSan?) and really need > to be compiled properly, otherwise you?ll end up with false positives. > The matrix of builds is REALLY hard to track. Each time we update Boost, we had to update > a lot of things. > I tried using ExternalProject and use b2 (build tool from Boost) to build it and instead of having > lots of build jobs with a mirror of the flags, you end up mirroring the flags in your CMake files > instead, which is still not good enough. > > In the end, I looked at how Boost is actually built. And for most libraries, it?s plain simple. > A static library with a few files, some define, sometimes a platform specific source file. > What if instead of having an external build tool, I built it from CMake instead? > It would propagate all the build flags, target, instrumentation and compiler information from the main > build to it and just work. > I tried it and it worked in no time! We replaced our Boost 1.59 binary distribution with the source > distribution and it?s much easier. When people build our library for a different target, they don?t have > to download new binaries, they just reuse the same sources. > Later on, we found a bug in Boost 1.59 (fixed in later versions) and patched it. We updated our source > bundle and everything was smooth. > Much later on, we wanted to use 1.61. We just updated the source bundle again, the list of source > files or compilation flags for the libraries we use didn?t change. It was again effortless. > > Overall, building boost takes 10s on our developers? machines. The sources aren?t changed often, > so the cost is pretty low. It needs attention when we upgrade it, but that?s quite rare. > > We try now to use the same approach for other libraries when we add them. Some of them are > already using CMake and it?s somewhat easier, but since most people still target version 2.8 (or 2.6...), > we find it better to rewrite the build scripts ourselves and use modern features (as in, everything is > a target that propagates requirements, we don?t propagate variables). > It makes it also much easier to build a library for another platform that wasn?t targeted by the original > project. > > If people are interested, I could share the CMakeLists.txt file we use for Boost. It doesn?t build all > the libraries (some are hard like Context) and uses some internal macros, but it should be plain > simple to tweak for your use. > > /Florent > > > On 12 Aug 2016, at 21:59, Robert Dailey > wrote: > > > > Hello, > > > > There is an internal C++ product at the company I work for which I > > have written a series of CMake scripts for. This project actually has > > dependencies on several open source libraries, such as boost, > > freetype, openssl, etc. > > > > Right now what we do is build each of these third party libraries *by > > hand*, once for every platform we support (Windows, Linux x86, Android > > NDK). Then we stuff the includes (headers) and libraries > > (static/shared) in a submodule and the primary code base's CMake > > scripts pull them in as interface targets. > > > > This works well and is light-weight but is a pain when upgrading or > > changing libraries. It's a pain because if I want to upgrade boost, I > > have to build it up to 6 times (once for each platform and once for > > each configuration). > > > > I've been thinking of a different approach for a while. I've done some > > toying around with the "Super Build" concept, where I have a separate > > CMake project that does nothing but use the ExternalProject module to > > build libraries in real time along with our project. So the order of > > operations would be as follows (for our automated build server): > > > > 1. Clone our "Third Party" repository > > 2. Use CMake to generate & build the "Super Build" project (this > > builds boost, openssl, freetype, etc for the current platform). > > 3. Clone the main code base's repository > > 4. Use CMake to generate & build, using find_package() to refer to > > interface targets exported by those third party libraries built in > > step 2 > > > > Obviously this will make builds take significantly longer, because > > we're constantly rebuilding the same third party libraries over and > > over again. However, it virtually eliminates the maintenance burden > > for third party libraries because they are built inherently with > > everything else. > > > > Note that I can't refer to pre-built libraries in our build > > environments because we need very specific control over the versions > > of our libraries as well as the toolchains that were used to build > > them. Also we may specifically build our libraries a certain way (such > > as boost). For this reason we do not rely on our external environment > > or external package managers to fulfill third party dependencies, like > > most open source projects do on Linux for example. > > > > Does this "Super Build" approach sound like a better idea? What other > > options are available? The downside with the "Super Build" solution is > > that it will become very difficult to make the transition between > > building third party and building our code base seamless. I can't do > > both in the same generate step because find_package() can't be called > > until the libraries are built & installed. > > -- > > > > Powered by www.kitware.com > > > > Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ > > > > Kitware offers various services to support the CMake community. For more information on each offering, please visit: > > > > CMake Support: http://cmake.org/cmake/help/support.html > > CMake Consulting: http://cmake.org/cmake/help/consulting.html > > CMake Training Courses: http://cmake.org/cmake/help/training.html > > > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html > > > > Follow this link to subscribe/unsubscribe: > > http://public.kitware.com/mailman/listinfo/cmake-developers > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > > > -- > Benjamin BALLET > Ing?nieur R&D > > ACTIVISU > 19, rue Klock - 92110 Clichy > > Standard T?l : 01 44 69 37 37 > > www.activisu.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From florent.castelli at gmail.com Tue Aug 16 09:37:59 2016 From: florent.castelli at gmail.com (Florent Castelli) Date: Tue, 16 Aug 2016 15:37:59 +0200 Subject: [cmake-developers] Need ideas/opinions on third party library management In-Reply-To: References: <1D3F7CC1-58CE-4B01-A2E2-9AFBB746B384@gmail.com> Message-ID: Well, I tried upstreaming the new build scripts to some projects and it didn?t go well. Some of the reasons I?ve heard of: - Windows developpers don?t use CMake, they have project files on the repository. The CMake files for Windows will never be updated. - I installed CMake 2.8.6 five years ago and I don?t want to update yet again! People relying on old versions is quite common and any attempt to raise the min version will be frowned upon (see the discussion in the LLVM mailing lists for example). - We prefer to use autotools and don?t want to have to learn CMake. That?s fair. But also, no one likes to build an autotools backed project for Android or iOS. Rewriting build scripts for all libraries isn?t trivial. But depending on the scale of your organization, it might make sense to spend the time. Google is doing something very similar by writing Blaze build scripts for the *world*. They can leverage all their infrastructure for fast and correct builds then. In a similar way here, we just have our compilation options set once in our toolchain file and that?s it, it works correctly, without duplicating jobs to build the libraries. Also, everything is built from the tip of the tree, which is a different software practice than using set releases. It?s not for everyone though. As for using a heterogenous build system, I?m really against it. Even if they work for the platform you are trying to target, you may run into other issues. Some build systems are really badly designed and will slowdown your iterations a lot. Most people here will use Ninja and have a very fast or instant ?no-op? build. You may forget to keep options in sync. Or even might not be able to pass some options to the other build system. I?ve never heard of Spack before. It looks better than other solutions I?ve seen before. But you still have to manage all the options from your build script and publish the binaries somewhere. Then you need to teach your build scripts to get the right version. I won?t trade my builds from source for a set of prebuilt binaries anytime soon I think :) /Florent > On 16 Aug 2016, at 14:52, Elizabeth A. Fischer wrote: > > CMake builds for existing libraries are certainly an interesting and useful thing, and deserve to be posted in a GitHub repo somewhere. They should also serve as the basis of a campaign to get the library authors to incorporate the CMake build directly in their repos. > > But any approach that requires every build to be ported to CMake will be difficult and labor-prone to scale. Writing a meta-build recipe is usually much easier. > > Spack handles the combinatorial dependencies you mention in a sophisticated, graceful way that most meta-builders do not. Its only problem is it does not (yet) run on Windows. There's no fundamental reason why not; we just need someone to get involved and start trying it on Windows. > > -- Elizabeth > > > On Tue, Aug 16, 2016 at 6:52 AM, Florent Castelli > wrote: > At Spotify, we use CMake a lot for our large C++ library shared by all the clients. > After trying to build libraries for each platform and variant, we basically gave up and we now > use a super-build approach. > > For example, Boost is used by 5 platforms: Windows, OSX, Linux, Android and iOS. > Each platform has a different CPU target (or many 32/64bit, x86/ARM). > Each platform has many compilers. > Some platforms have instrumentation options (Debug / Release, ASan, MSan?) and really need > to be compiled properly, otherwise you?ll end up with false positives. > The matrix of builds is REALLY hard to track. Each time we update Boost, we had to update > a lot of things. > I tried using ExternalProject and use b2 (build tool from Boost) to build it and instead of having > lots of build jobs with a mirror of the flags, you end up mirroring the flags in your CMake files > instead, which is still not good enough. > > In the end, I looked at how Boost is actually built. And for most libraries, it?s plain simple. > A static library with a few files, some define, sometimes a platform specific source file. > What if instead of having an external build tool, I built it from CMake instead? > It would propagate all the build flags, target, instrumentation and compiler information from the main > build to it and just work. > I tried it and it worked in no time! We replaced our Boost 1.59 binary distribution with the source > distribution and it?s much easier. When people build our library for a different target, they don?t have > to download new binaries, they just reuse the same sources. > Later on, we found a bug in Boost 1.59 (fixed in later versions) and patched it. We updated our source > bundle and everything was smooth. > Much later on, we wanted to use 1.61. We just updated the source bundle again, the list of source > files or compilation flags for the libraries we use didn?t change. It was again effortless. > > Overall, building boost takes 10s on our developers? machines. The sources aren?t changed often, > so the cost is pretty low. It needs attention when we upgrade it, but that?s quite rare. > > We try now to use the same approach for other libraries when we add them. Some of them are > already using CMake and it?s somewhat easier, but since most people still target version 2.8 (or 2.6...), > we find it better to rewrite the build scripts ourselves and use modern features (as in, everything is > a target that propagates requirements, we don?t propagate variables). > It makes it also much easier to build a library for another platform that wasn?t targeted by the original > project. > > If people are interested, I could share the CMakeLists.txt file we use for Boost. It doesn?t build all > the libraries (some are hard like Context) and uses some internal macros, but it should be plain > simple to tweak for your use. > > /Florent > > > On 12 Aug 2016, at 21:59, Robert Dailey > wrote: > > > > Hello, > > > > There is an internal C++ product at the company I work for which I > > have written a series of CMake scripts for. This project actually has > > dependencies on several open source libraries, such as boost, > > freetype, openssl, etc. > > > > Right now what we do is build each of these third party libraries *by > > hand*, once for every platform we support (Windows, Linux x86, Android > > NDK). Then we stuff the includes (headers) and libraries > > (static/shared) in a submodule and the primary code base's CMake > > scripts pull them in as interface targets. > > > > This works well and is light-weight but is a pain when upgrading or > > changing libraries. It's a pain because if I want to upgrade boost, I > > have to build it up to 6 times (once for each platform and once for > > each configuration). > > > > I've been thinking of a different approach for a while. I've done some > > toying around with the "Super Build" concept, where I have a separate > > CMake project that does nothing but use the ExternalProject module to > > build libraries in real time along with our project. So the order of > > operations would be as follows (for our automated build server): > > > > 1. Clone our "Third Party" repository > > 2. Use CMake to generate & build the "Super Build" project (this > > builds boost, openssl, freetype, etc for the current platform). > > 3. Clone the main code base's repository > > 4. Use CMake to generate & build, using find_package() to refer to > > interface targets exported by those third party libraries built in > > step 2 > > > > Obviously this will make builds take significantly longer, because > > we're constantly rebuilding the same third party libraries over and > > over again. However, it virtually eliminates the maintenance burden > > for third party libraries because they are built inherently with > > everything else. > > > > Note that I can't refer to pre-built libraries in our build > > environments because we need very specific control over the versions > > of our libraries as well as the toolchains that were used to build > > them. Also we may specifically build our libraries a certain way (such > > as boost). For this reason we do not rely on our external environment > > or external package managers to fulfill third party dependencies, like > > most open source projects do on Linux for example. > > > > Does this "Super Build" approach sound like a better idea? What other > > options are available? The downside with the "Super Build" solution is > > that it will become very difficult to make the transition between > > building third party and building our code base seamless. I can't do > > both in the same generate step because find_package() can't be called > > until the libraries are built & installed. > > -- > > > > Powered by www.kitware.com > > > > Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ > > > > Kitware offers various services to support the CMake community. For more information on each offering, please visit: > > > > CMake Support: http://cmake.org/cmake/help/support.html > > CMake Consulting: http://cmake.org/cmake/help/consulting.html > > CMake Training Courses: http://cmake.org/cmake/help/training.html > > > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html > > > > Follow this link to subscribe/unsubscribe: > > http://public.kitware.com/mailman/listinfo/cmake-developers > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Tue Aug 16 14:09:13 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 16 Aug 2016 14:09:13 -0400 Subject: [cmake-developers] [PATCH] cmFileCommand: Download continuation support In-Reply-To: <211201471345908@web22j.yandex.ru> References: <211201471345908@web22j.yandex.ru> Message-ID: <0dc663c5-5d1f-313d-061e-c6ad4c449448@kitware.com> On 08/16/2016 07:11 AM, Titov Denis wrote: > This patch introduces several options for the file(DOWNLOAD ...) command, which > would be useful in case of dealing with big files or unstable network connections. Nice, thanks. Good start. Here are a few minor comments. > The added options are: Please also update `Help/command/file.rst` to document these. > +void crossplatform_sleep(int delay_s) Please use cmSystemTools::Delay instead. Thanks, -Brad From brad.king at kitware.com Tue Aug 16 14:09:18 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 16 Aug 2016 14:09:18 -0400 Subject: [cmake-developers] cmake -E capabilities [v3] In-Reply-To: References: Message-ID: On 08/15/2016 09:56 AM, Tobias Hunger wrote: > here is a new update of the cmake -E capabilities patch. Thanks. Applied with minor revisions and an additional test: cmake: Add `cmake -E capabilities` mode https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=49ad7f9a -Brad From brad.king at kitware.com Tue Aug 16 14:09:31 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 16 Aug 2016 14:09:31 -0400 Subject: [cmake-developers] llvm coverage format .profdata In-Reply-To: <3D3B96AB-F158-4C2C-9424-3EB59C85E380@rogue-research.com> References: <3D3B96AB-F158-4C2C-9424-3EB59C85E380@rogue-research.com> Message-ID: <4b5720d2-dd30-b9c5-66d0-a4c3eb08d491@kitware.com> On 08/15/2016 02:24 PM, Alex Ciobanu wrote: > Are there any plans in this direction? Not currently, but one can look at the implementations of the currently supported formats to add it. Thanks, -Brad From brad.king at kitware.com Tue Aug 16 14:09:44 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 16 Aug 2016 14:09:44 -0400 Subject: [cmake-developers] [PATCH v7] For Windows encode process output to internally used encoding In-Reply-To: <20160815203421.28064-1-davispuh@gmail.com> References: <20160815203421.28064-1-davispuh@gmail.com> Message-ID: <9b10aee6-64fe-55dc-17af-d0fcd6c37a00@kitware.com> On 08/15/2016 04:34 PM, D?vis Mos?ns wrote: > Typically Windows applications (eg. MSVC compiler) use current console's > codepage for output to pipes so we need to encode that to internally used > encoding (KWSYS_ENCODING_DEFAULT_CODEPAGE). Thanks. Applied with minor tweaks and merged to `next` for testing: Windows: Encode child process output to internally-used encoding https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=0a818239 -Brad From tobias.hunger at gmail.com Tue Aug 16 14:39:13 2016 From: tobias.hunger at gmail.com (Tobias Hunger) Date: Tue, 16 Aug 2016 20:39:13 +0200 Subject: [cmake-developers] cmake -E capabilities [v3] In-Reply-To: References: Message-ID: Hi Brad! Am 16.08.2016 20:09 schrieb "Brad King" : > > On 08/15/2016 09:56 AM, Tobias Hunger wrote: > > here is a new update of the cmake -E capabilities patch. > > Thanks. Applied with minor revisions and an additional test: > > cmake: Add `cmake -E capabilities` mode > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=49ad7f9a Thanks for cleaning up after me! Did you make any progress with libuv? I would love to start pushing the server-mode patches. All the other pieces are in place already! Best Regards, Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Tue Aug 16 14:43:43 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 16 Aug 2016 14:43:43 -0400 Subject: [cmake-developers] cmake -E capabilities [v3] In-Reply-To: References: Message-ID: On 08/16/2016 02:39 PM, Tobias Hunger wrote: > Did you make any progress with libuv? I would love to start pushing > the server-mode patches. All the other pieces are in place already! Great. I haven't gotten a chance to import libuv yet. Now that you're ready for it I'll try to raise priority on that a bit. -Brad From alexander.lamaison at gmail.com Tue Aug 16 16:44:43 2016 From: alexander.lamaison at gmail.com (Alexander Lamaison) Date: Tue, 16 Aug 2016 21:44:43 +0100 Subject: [cmake-developers] FindOpenSSL: Link libraries needed for static OpenSSL. In-Reply-To: References: Message-ID: On 15 August 2016 at 14:25, Brad King wrote: > On 08/14/2016 01:58 PM, Alexander Lamaison wrote: >> This patch updates FindOpenSSL.cmake to include the necessary system >> libs in the OpenSSL libraries variables and in the OpenSSL::Crypto and >> OpenSSL::SSL target properties. > > Thanks. The singluar-named OPENSSL_{SSL,CRYPTO}_LIBRARY variables > are meant just to hold the specific library files (i.e. the result > of one find_library call). Instead of placing the dependencies > directly in them we should have other (internal) variables to hold > them. That will also avoid duplicating conditions for filling in > the imported target information. I've attached a new patch without the changes to the singular-named variables. Alex -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-FindOpenSSL-Link-libraries-needed-for-static-OpenSSL.patch Type: text/x-patch Size: 3555 bytes Desc: not available URL: From jediry01 at yahoo.com Tue Aug 16 18:43:48 2016 From: jediry01 at yahoo.com (Ryan Saunders) Date: Tue, 16 Aug 2016 22:43:48 +0000 (UTC) Subject: [cmake-developers] Appx generator for CPack? References: <131596044.15387901.1471387428986.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <131596044.15387901.1471387428986.JavaMail.yahoo@mail.yahoo.com> Hi CMake people! I notice that there is no CPack generator for Microsoft's .appx package format. I would like there to be. It seems pretty straightforward to do...basically just get all the files into the right directory structure (similarly to the archive generators, I think), and then call out to Visual Studio's MakeAppx.exe to create the package. Two questions:1. Is anyone already working on this? If not, then I'll attempt it.2. Am I missing anything? Any "gotchas" I should worry about? Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From davispuh at gmail.com Wed Aug 17 06:11:40 2016 From: davispuh at gmail.com (=?UTF-8?B?RMSBdmlzIE1vc8SBbnM=?=) Date: Wed, 17 Aug 2016 13:11:40 +0300 Subject: [cmake-developers] [PATCH v7] For Windows encode process output to internally used encoding In-Reply-To: <9b10aee6-64fe-55dc-17af-d0fcd6c37a00@kitware.com> References: <20160815203421.28064-1-davispuh@gmail.com> <9b10aee6-64fe-55dc-17af-d0fcd6c37a00@kitware.com> Message-ID: 2016-08-16 21:09 GMT+03:00 Brad King : > On 08/15/2016 04:34 PM, D?vis Mos?ns wrote: >> Typically Windows applications (eg. MSVC compiler) use current console's >> codepage for output to pipes so we need to encode that to internally used >> encoding (KWSYS_ENCODING_DEFAULT_CODEPAGE). > > Thanks. Applied with minor tweaks and merged to `next` for testing: > > Windows: Encode child process output to internally-used encoding > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=0a818239 Great! Thanks! From brad.king at kitware.com Wed Aug 17 08:53:48 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 17 Aug 2016 08:53:48 -0400 Subject: [cmake-developers] Appx generator for CPack? In-Reply-To: <131596044.15387901.1471387428986.JavaMail.yahoo@mail.yahoo.com> References: <131596044.15387901.1471387428986.JavaMail.yahoo.ref@mail.yahoo.com> <131596044.15387901.1471387428986.JavaMail.yahoo@mail.yahoo.com> Message-ID: On 08/16/2016 06:43 PM, Ryan Saunders via cmake-developers wrote: > I notice that there is no CPack generator for Microsoft's .appx > package format. > > 1. Is anyone already working on this? If not, then I'll attempt it. > 2. Am I missing anything? Any "gotchas" I should worry about? I'm not aware of anyone working on this. Please go ahead if you wish. Thanks, -Brad From brad.king at kitware.com Wed Aug 17 08:53:54 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 17 Aug 2016 08:53:54 -0400 Subject: [cmake-developers] FindOpenSSL: Link libraries needed for static OpenSSL. In-Reply-To: References: Message-ID: <75b45b45-d217-be9f-3656-1fd10b7a1cf9@kitware.com> On 08/16/2016 04:44 PM, Alexander Lamaison wrote: > I've attached a new patch without the changes to the singular-named variables. Thanks. > + if(WIN32 AND NOT CYGWIN) > + set_target_properties(OpenSSL::Crypto PROPERTIES > + INTERFACE_LINK_LIBRARIES crypt32) > + else() > + set_target_properties(OpenSSL::Crypto PROPERTIES > + INTERFACE_LINK_LIBRARIES "${CMAKE_DL_LIBS}") > + endif() The addition of crypt32/dl is done with this logic in several places. Instead please use a _OPENSSL_LIBRARY_DEPENDS variable to hold the value and then reference it in the other locations so that the conditions for setting it do not have to be duplicated. Thanks, -Brad From brad.king at kitware.com Wed Aug 17 09:47:40 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 17 Aug 2016 09:47:40 -0400 Subject: [cmake-developers] [PATCH v7] For Windows encode process output to internally used encoding In-Reply-To: References: <20160815203421.28064-1-davispuh@gmail.com> <9b10aee6-64fe-55dc-17af-d0fcd6c37a00@kitware.com> Message-ID: <5f7f6a2b-7da8-d74f-e46d-f54941dd6cd0@kitware.com> On 08/17/2016 06:11 AM, D?vis Mos?ns wrote: >> Applied with minor tweaks and merged to `next` for testing: > > Great! Thanks! I squashed in one warning fix: Windows: Encode child process output to internally-used encoding https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=12924660 However, then I had to revert the change from `next` because it causes the CTest.UpdateGIT test to fail on Windows machines. I think the problem is that we run Git with a `-z` option to produce binary output. In such cases we should not do any encoding conversions. cmProcessTools and RunSingleCommand will need to gain options for this. Also I noticed that if DecodeText buffers partial characters we may need a finalize step later to finish them off. Otherwise invalid byte sequences may be dropped if they appear at the end. Please fetch the above version and revise it as needed. Thanks, -Brad From eike at sf-mail.de Wed Aug 17 10:04:19 2016 From: eike at sf-mail.de (Rolf Eike Beer) Date: Wed, 17 Aug 2016 16:04:19 +0200 Subject: [cmake-developers] FindOpenSSL: Link libraries needed for static OpenSSL. In-Reply-To: <75b45b45-d217-be9f-3656-1fd10b7a1cf9@kitware.com> References: <75b45b45-d217-be9f-3656-1fd10b7a1cf9@kitware.com> Message-ID: <985c3b61ec002784afda5f1a6cb06d74@sf-mail.de> Am 2016-08-17 14:53, schrieb Brad King: > On 08/16/2016 04:44 PM, Alexander Lamaison wrote: >> I've attached a new patch without the changes to the singular-named >> variables. > > Thanks. > >> + if(WIN32 AND NOT CYGWIN) >> + set_target_properties(OpenSSL::Crypto PROPERTIES >> + INTERFACE_LINK_LIBRARIES crypt32) >> + else() >> + set_target_properties(OpenSSL::Crypto PROPERTIES >> + INTERFACE_LINK_LIBRARIES "${CMAKE_DL_LIBS}") >> + endif() > > The addition of crypt32/dl is done with this logic in several places. > Instead please use a _OPENSSL_LIBRARY_DEPENDS variable to hold the > value and then reference it in the other locations so that the > conditions for setting it do not have to be duplicated. Don't forget to unset() the variable at the end of the module. Greetings, Eike -- From aoe at google.com Wed Aug 17 16:48:23 2016 From: aoe at google.com (Chaoren Lin) Date: Wed, 17 Aug 2016 13:48:23 -0700 Subject: [cmake-developers] [PATCH] Tests: Refactor testing of extra generators. Message-ID: <1471466903-17145-1-git-send-email-aoe@google.com> From: Chaoren Lin Use a loop instead of repeating the same thing multiple times. --- Tests/CMakeLists.txt | 77 ++++++++++++++++++++-------------------------------- 1 file changed, 30 insertions(+), 47 deletions(-) diff --git a/Tests/CMakeLists.txt b/Tests/CMakeLists.txt index 6f1805b..c119cfd 100644 --- a/Tests/CMakeLists.txt +++ b/Tests/CMakeLists.txt @@ -569,60 +569,43 @@ if(BUILD_TESTING) ADD_LINK_FLAGS_TEST(mod_flags_config dll_flags_config) ADD_LINK_FLAGS_TEST(exe_flags_config mod_flags_config) - # If we are running right now with a UnixMakefiles based generator, + # If we are running right now with a Unix Makefiles or Ninja based generator, # build the "Simple" test with the ExtraGenerators, if available # This doesn't test whether the generated project files work (unfortunately), # mainly it tests that cmake doesn't crash when generating these project files. - if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles" OR ${CMAKE_GENERATOR} MATCHES "KDevelop") + if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles" OR ${CMAKE_GENERATOR} MATCHES "Ninja") + # check which generators we have execute_process(COMMAND ${CMAKE_CMAKE_COMMAND} --help OUTPUT_VARIABLE cmakeOutput ERROR_VARIABLE cmakeOutput) - # check for the Eclipse generator - if ("${cmakeOutput}" MATCHES Eclipse) - add_test(Simple_EclipseGenerator ${CMAKE_CTEST_COMMAND} - --build-and-test - "${CMake_SOURCE_DIR}/Tests/Simple" - "${CMake_BINARY_DIR}/Tests/Simple_EclipseGenerator" - --build-two-config - --build-generator "Eclipse CDT4 - Unix Makefiles" - --build-generator-platform "${CMAKE_GENERATOR_PLATFORM}" - --build-generator-toolset "${CMAKE_GENERATOR_TOOLSET}" - --build-project Simple - --build-options ${build_options} - --test-command Simple) - list(APPEND TEST_BUILD_DIRS "${CMake_BINARY_DIR}/Tests/Simple_EclipseGenerator") - endif () - # check for the CodeBlocks generator - if ("${cmakeOutput}" MATCHES CodeBlocks) - add_test(Simple_CodeBlocksGenerator ${CMAKE_CTEST_COMMAND} - --build-and-test - "${CMake_SOURCE_DIR}/Tests/Simple" - "${CMake_BINARY_DIR}/Tests/Simple_CodeBlocksGenerator" - --build-two-config - --build-generator "CodeBlocks - Unix Makefiles" - --build-generator-platform "${CMAKE_GENERATOR_PLATFORM}" - --build-generator-toolset "${CMAKE_GENERATOR_TOOLSET}" - --build-project Simple - --build-options ${build_options} - --test-command Simple) - list(APPEND TEST_BUILD_DIRS "${CMake_BINARY_DIR}/Tests/Simple_CodeBlocksGenerator") - endif () - # check for the KDevelop3 generator - if ("${cmakeOutput}" MATCHES KDevelop3) - add_test(Simple_KDevelop3Generator ${CMAKE_CTEST_COMMAND} - --build-and-test - "${CMake_SOURCE_DIR}/Tests/Simple" - "${CMake_BINARY_DIR}/Tests/Simple_KDevelop3Generator" - --build-two-config - --build-generator "KDevelop3 - Unix Makefiles" - --build-generator-platform "${CMAKE_GENERATOR_PLATFORM}" - --build-generator-toolset "${CMAKE_GENERATOR_TOOLSET}" - --build-project Simple - --build-options ${build_options} - --test-command Simple) - list(APPEND TEST_BUILD_DIRS "${CMake_BINARY_DIR}/Tests/Simple_KDevelop3Generator") - endif () + set(extraGenerators + "CodeBlocks" + "CodeLite" + "Eclipse CDT4" + "Kate" + "KDevelop3" + "Sublime Text 2") + + foreach(extraGenerator ${extraGenerators}) + if ("${cmakeOutput}" MATCHES "${extraGenerator} - ${CMAKE_GENERATOR}") + set(extraGeneratorTestName "Simple_${extraGenerator}Generator") + string(REPLACE " " "" extraGeneratorTestName ${extraGeneratorTestName}) + + add_test(${extraGeneratorTestName} ${CMAKE_CTEST_COMMAND} + --build-and-test + "${CMake_SOURCE_DIR}/Tests/Simple" + "${CMake_BINARY_DIR}/Tests/${extraGeneratorTestName}" + --build-two-config + --build-generator "${extraGenerator} - ${CMAKE_GENERATOR}" + --build-generator-platform "${CMAKE_GENERATOR_PLATFORM}" + --build-generator-toolset "${CMAKE_GENERATOR_TOOLSET}" + --build-project Simple + --build-options ${build_options} + --test-command Simple) + list(APPEND TEST_BUILD_DIRS "${CMake_BINARY_DIR}/Tests/${extraGeneratorTestName}") + endif () + endforeach(extraGenerator) endif() -- 2.8.0.rc3.226.g39d4020 From costean_adrian at yahoo.com Thu Aug 18 04:11:43 2016 From: costean_adrian at yahoo.com (Costean Adrian) Date: Thu, 18 Aug 2016 08:11:43 +0000 (UTC) Subject: [cmake-developers] Dependency mechanism of CMake References: <2146442261.17295270.1471507903279.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <2146442261.17295270.1471507903279.JavaMail.yahoo@mail.yahoo.com> Hello CMake developers, I've been trying to figure out how CMake works ( so far no chance, the documentation of "behind the scenes of CMake" is kind of limited). I am interested in how CMake calculates the dependencies, if it does it efficient and correct. Could anyone brief me with a quick overview? Thanks and regards, Adrian C. From jeanmichael.celerier at gmail.com Thu Aug 18 04:56:29 2016 From: jeanmichael.celerier at gmail.com (=?UTF-8?Q?Jean=2DMicha=C3=ABl_Celerier?=) Date: Thu, 18 Aug 2016 10:56:29 +0200 Subject: [cmake-developers] Dependency mechanism of CMake In-Reply-To: <2146442261.17295270.1471507903279.JavaMail.yahoo@mail.yahoo.com> References: <2146442261.17295270.1471507903279.JavaMail.yahoo.ref@mail.yahoo.com> <2146442261.17295270.1471507903279.JavaMail.yahoo@mail.yahoo.com> Message-ID: Hi, Did you read this page ? The "Build Specification & Usage Requirements" section clearly explains how it works imho. https://cmake.org/cmake/help/v3.6/manual/cmake-buildsystem.7.html and https://cmake.org/cmake/help/v3.6/manual/cmake-buildsystem.7.html#build-specification-and-usage-requirements Best Jean-Micha?l On Thu, Aug 18, 2016 at 10:11 AM, Costean Adrian via cmake-developers < cmake-developers at cmake.org> wrote: > Hello CMake developers, > I've been trying to figure out how CMake works ( so far no chance, the > documentation of "behind the scenes of CMake" is kind of limited). > I am interested in how CMake calculates the dependencies, if it does it > efficient and correct. > Could anyone brief me with a quick overview? > Thanks and regards, > Adrian C. > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/ > opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel at pfeifer-mail.de Thu Aug 18 05:37:55 2016 From: daniel at pfeifer-mail.de (Daniel Pfeifer) Date: Thu, 18 Aug 2016 11:37:55 +0200 Subject: [cmake-developers] variables for top level project version Message-ID: Hi! The project() command currently sets some variables in both PROJECT_ and _ form. So far good. It also sets CMAKE_PROJECT_NAME to the top level project name. This is not documented. It does not set any other variables of the form CMAKE_PROJECT_. This would be useful for the version variables. CPack.cmake initializes CPACK_PACKAGE_NAME with ${CMAKE_PROJECT_NAME}, but it initializes CPACK_PACKAGE_VERSION with "0.1.1". It should initialize all version variables with the values from the top level project() command. Cheers, Daniel From kernelmd at yandex.ru Thu Aug 18 13:47:48 2016 From: kernelmd at yandex.ru (Titov Denis) Date: Thu, 18 Aug 2016 20:47:48 +0300 Subject: [cmake-developers] Subject: [PATCH v2] cmFileCommand: Download continuation support Message-ID: <320021471542468@web26m.yandex.ru> This patch introduces several options for the file(DOWNLOAD ...) command, which would be useful in case of dealing with big files or unstable network connections. The added options are: RETRY_COUNT -- sets maximal amount of download restarts, default value: 1 RETRY_DELAY -- sets delay before restarting download (in seconds), default value: 0.0 RETRY_MAX_TIME -- sets maximal time spent in downloading file (in seconds), default value: infinity RETRY_CONTINUE -- if set, makes cmake try to continue downloading of the existing chunk, instead of discarding it and starting all over. This option is not set by default Notes: The RETRY_CONTINUE option requires server-side support of http partial get (content-range header). Unfortunately, I haven't been able to properly test the RETRY_CONTINUE option, as I didn't have access to the appropriate server. Any help in this area is encouraged. --- Help/command/file.rst | 17 +++ Source/cmFileCommand.cxx | 271 ++++++++++++++++++++++++++++++++--------------- 2 files changed, 205 insertions(+), 83 deletions(-) diff --git a/Help/command/file.rst b/Help/command/file.rst index 256d16d..f1095b7 100644 --- a/Help/command/file.rst +++ b/Help/command/file.rst @@ -240,6 +240,23 @@ Additional options to ``DOWNLOAD`` are: ``TLS_CAINFO `` Specify a custom Certificate Authority file for ``https://`` URLs. +``RETRY_COUNT `` + Set maximal amount of download restarts, default value: 1 + +``RETRY_DELAY `` + Set delay before restarting download, default value: 0.0 + +``RETRY_MAX_TIME `` + Set maximal time spent in downloading file, default value: infinity + +``RETRY_CONTINUE`` + If set, makes cmake try to continue downloading of the existing chunk, + instead of discarding it and starting all over. This option is not set + by default. + + Note that this option requires server-side support of http partial get + (content-range header). + For ``https://`` URLs CMake must be built with OpenSSL support. ``TLS/SSL`` certificates are not checked by default. Set ``TLS_VERIFY`` to ``ON`` to check certificates and/or use ``EXPECTED_HASH`` to verify downloaded content. diff --git a/Source/cmFileCommand.cxx b/Source/cmFileCommand.cxx index 835b118..bbe8839 100644 --- a/Source/cmFileCommand.cxx +++ b/Source/cmFileCommand.cxx @@ -34,6 +34,9 @@ // include sys/stat.h after sys/types.h #include +#include +#include + #include #include #include @@ -2481,6 +2484,11 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) std::string hashMatchMSG; CM_AUTO_PTR hash; bool showProgress = false; + int retryMaxCount = 1; + double retryDelayS = 0.0; + double retryMaxTimeS = DBL_MAX; + bool retryContinue = false; + cmsys::ofstream fout; while (i != args.end()) { if (*i == "TIMEOUT") { @@ -2564,7 +2572,34 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) return false; } hashMatchMSG = algo + " hash"; + } else if (*i == "RETRY_COUNT") { + ++i; + if (i != args.end()) { + retryMaxCount = atoi(i->c_str()); + } else { + this->SetError("DOWNLOAD missing count for RETRY_COUNT"); + return false; + } + } else if (*i == "RETRY_DELAY") { + ++i; + if (i != args.end()) { + retryDelayS = atof(i->c_str()); + } else { + this->SetError("DOWNLOAD missing time for RETRY_DELAY"); + return false; + } + } else if (*i == "RETRY_MAX_TIME") { + ++i; + if (i != args.end()) { + retryMaxTimeS = atof(i->c_str()); + } else { + this->SetError("DOWNLOAD missing time for RETRY_MAX_TIME"); + return false; + } + } else if (*i == "RETRY_CONTINUE") { + retryContinue = true; } + ++i; } // If file exists already, and caller specified an expected md5 or sha, @@ -2599,110 +2634,171 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) return false; } - cmsys::ofstream fout(file.c_str(), std::ios::binary); - if (!fout) { - this->SetError("DOWNLOAD cannot open file for write."); - return false; - } - #if defined(_WIN32) && defined(CMAKE_ENCODING_UTF8) url = fix_file_url_windows(url); #endif + cmFileCommandVectorOfChar chunkDebug; + ::CURL* curl; ::curl_global_init(CURL_GLOBAL_DEFAULT); - curl = ::curl_easy_init(); - if (!curl) { - this->SetError("DOWNLOAD error initializing curl."); - return false; - } - cURLEasyGuard g_curl(curl); - ::CURLcode res = ::curl_easy_setopt(curl, CURLOPT_URL, url.c_str()); - check_curl_result(res, "DOWNLOAD cannot set url: "); + ::CURLcode res; + int tries = 0; + double elapsed = 0.0; + time_t start, end; + while (tries < retryMaxCount && elapsed <= retryMaxTimeS) { + ++tries; + time(&start); + + curl = ::curl_easy_init(); + if (!curl) { + this->SetError("DOWNLOAD error initializing curl."); + ::curl_global_cleanup(); + return false; + } - // enable HTTP ERROR parsing - res = ::curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1); - check_curl_result(res, "DOWNLOAD cannot set http failure option: "); + if (cmSystemTools::FileExists(file.c_str())) { // Something was downloaded. + // Check hash. + if (hash.get()) { + std::string actualHash = hash->HashFile(file); + if (actualHash == expectedHash) { // File is complete, exit. + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return true; + } + } - res = ::curl_easy_setopt(curl, CURLOPT_USERAGENT, "curl/" LIBCURL_VERSION); - check_curl_result(res, "DOWNLOAD cannot set user agent option: "); + if (retryContinue == false) { // Discard downloaded chunk. + fout.open(file.c_str(), std::ios::binary | std::ios::trunc); + if (!fout.good()) { + this->SetError("Cannot open file for writing"); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } + } else { // Try to continue. + fout.open(file.c_str(), std::ios::binary | std::ios::app); + if (!fout.good()) { + this->SetError("Cannot open file for writing"); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } + curl_easy_setopt(curl, CURLOPT_RESUME_FROM_LARGE, fout.tellp()); + } + } else { // Create new file. + fout.open(file.c_str(), std::ios::binary); + if (!fout.good()) { + this->SetError("Cannot open file for writing"); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } + } - res = ::curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, cmWriteToFileCallback); - check_curl_result(res, "DOWNLOAD cannot set write function: "); + cURLEasyGuard g_curl(curl); + res = ::curl_easy_setopt(curl, CURLOPT_URL, url.c_str()); + check_curl_result(res, "DOWNLOAD cannot set url: "); - res = ::curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, - cmFileCommandCurlDebugCallback); - check_curl_result(res, "DOWNLOAD cannot set debug function: "); + // enable HTTP ERROR parsing + res = ::curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1); + check_curl_result(res, "DOWNLOAD cannot set http failure option: "); - // check to see if TLS verification is requested - if (tls_verify) { - res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1); - check_curl_result(res, "Unable to set TLS/SSL Verify on: "); - } else { - res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0); - check_curl_result(res, "Unable to set TLS/SSL Verify off: "); - } - // check to see if a CAINFO file has been specified - // command arg comes first - std::string const& cainfo_err = cmCurlSetCAInfo(curl, cainfo); - if (!cainfo_err.empty()) { - this->SetError(cainfo_err); - return false; - } + res = ::curl_easy_setopt(curl, CURLOPT_USERAGENT, "curl/" LIBCURL_VERSION); + check_curl_result(res, "DOWNLOAD cannot set user agent option: "); - cmFileCommandVectorOfChar chunkDebug; + res = + ::curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, cmWriteToFileCallback); + check_curl_result(res, "DOWNLOAD cannot set write function: "); - res = ::curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void*)&fout); - check_curl_result(res, "DOWNLOAD cannot set write data: "); + res = ::curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, + cmFileCommandCurlDebugCallback); + check_curl_result(res, "DOWNLOAD cannot set debug function: "); - res = ::curl_easy_setopt(curl, CURLOPT_DEBUGDATA, (void*)&chunkDebug); - check_curl_result(res, "DOWNLOAD cannot set debug data: "); + // check to see if TLS verification is requested + if (tls_verify) { + res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1); + check_curl_result(res, "Unable to set TLS/SSL Verify on: "); + } else { + res = ::curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0); + check_curl_result(res, "Unable to set TLS/SSL Verify off: "); + } + // check to see if a CAINFO file has been specified + // command arg comes first + std::string const& cainfo_err = cmCurlSetCAInfo(curl, cainfo); + if (!cainfo_err.empty()) { + this->SetError(cainfo_err); + ::curl_easy_cleanup(curl); + ::curl_global_cleanup(); + return false; + } - res = ::curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); - check_curl_result(res, "DOWNLOAD cannot set follow-redirect option: "); + res = ::curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void*)&fout); + check_curl_result(res, "DOWNLOAD cannot set write data: "); - if (!logVar.empty()) { - res = ::curl_easy_setopt(curl, CURLOPT_VERBOSE, 1); - check_curl_result(res, "DOWNLOAD cannot set verbose: "); - } + res = ::curl_easy_setopt(curl, CURLOPT_DEBUGDATA, (void*)&chunkDebug); + check_curl_result(res, "DOWNLOAD cannot set debug data: "); - if (timeout > 0) { - res = ::curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout); - check_curl_result(res, "DOWNLOAD cannot set timeout: "); - } + res = ::curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); + check_curl_result(res, "DOWNLOAD cannot set follow-redirect option: "); - if (inactivity_timeout > 0) { - // Give up if there is no progress for a long time. - ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1); - ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, inactivity_timeout); - } + if (!logVar.empty()) { + res = ::curl_easy_setopt(curl, CURLOPT_VERBOSE, 1); + check_curl_result(res, "DOWNLOAD cannot set verbose: "); + } - // Need the progress helper's scope to last through the duration of - // the curl_easy_perform call... so this object is declared at function - // scope intentionally, rather than inside the "if(showProgress)" - // block... - // - cURLProgressHelper helper(this, "download"); + if (timeout > 0) { + res = ::curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout); + check_curl_result(res, "DOWNLOAD cannot set timeout: "); + } - if (showProgress) { - res = ::curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0); - check_curl_result(res, "DOWNLOAD cannot set noprogress value: "); + if (inactivity_timeout > 0) { + // Give up if there is no progress for a long time. + ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1); + ::curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, inactivity_timeout); + } - res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, - cmFileDownloadProgressCallback); - check_curl_result(res, "DOWNLOAD cannot set progress function: "); + // Need the progress helper's scope to last through the duration of + // the curl_easy_perform call... so this object is declared at loop + // scope intentionally, rather than inside the "if(showProgress)" + // block... + // + cURLProgressHelper helper(this, "download"); - res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, - reinterpret_cast(&helper)); - check_curl_result(res, "DOWNLOAD cannot set progress data: "); - } + if (showProgress) { + res = ::curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0); + check_curl_result(res, "DOWNLOAD cannot set noprogress value: "); - res = ::curl_easy_perform(curl); + res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, + cmFileDownloadProgressCallback); + check_curl_result(res, "DOWNLOAD cannot set progress function: "); - /* always cleanup */ - g_curl.release(); - ::curl_easy_cleanup(curl); + res = ::curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, + reinterpret_cast(&helper)); + check_curl_result(res, "DOWNLOAD cannot set progress data: "); + } + + res = ::curl_easy_perform(curl); + + /* always cleanup */ + g_curl.release(); + ::curl_easy_cleanup(curl); + fout.flush(); + fout.close(); + + // Download finished successfuly, exit the loop. + if (res == ::CURLE_OK) { + break; + // Server doesn't support content ranges... + } else if (retryContinue == true && res == ::CURLE_RANGE_ERROR) { + retryContinue = false; // Disable download continuation. + } + + cmUtils::Delay(retryDelayS * 1000); + time(&end); + elapsed += difftime(end, start); + } if (!statusVar.empty()) { std::ostringstream result; @@ -2712,10 +2808,19 @@ bool cmFileCommand::HandleDownloadCommand(std::vector const& args) ::curl_global_cleanup(); - // Explicitly flush/close so we can measure the md5 accurately. - // - fout.flush(); - fout.close(); + if (res != ::CURLE_OK) { + std::ostringstream oss; + // Failed by exhausting attempts + if (retryMaxCount != 1 && tries == retryMaxCount) { + oss << "Download failed after " << tries << " attempts. "; + } + // Failed by exhausting maximal time. + if (retryMaxTimeS < DBL_MAX && elapsed >= retryMaxTimeS) { + oss << "Download failed: time exhausted, " << elapsed << "s. spent. "; + } + oss << "Last CURL error: " << curl_easy_strerror(res); + return false; + } // Verify MD5 sum if requested: // -- 2.9.2 From steve.lorimer at gmail.com Thu Aug 18 16:37:48 2016 From: steve.lorimer at gmail.com (Steve Lorimer) Date: Thu, 18 Aug 2016 15:37:48 -0500 Subject: [cmake-developers] Unix makefile: concurrent build configurations Message-ID: Hello all New to this list, so apologies if it's been discussed before. I'd like to know why it's not possible to have both debug and release mode builds configured at the same time for Unix makefiles? I am aware that some generators (eg VisualStudio) do allow this behaviour. Why can I not have the same behaviour with Unix makefiles? TIA Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From egor.pugin at gmail.com Sat Aug 20 07:48:22 2016 From: egor.pugin at gmail.com (Egor Pugin) Date: Sat, 20 Aug 2016 14:48:22 +0300 Subject: [cmake-developers] Security in CMake Message-ID: Hi, I'm working on a package manager based on cmake. And some cmake instructions are downloaded with user packages. I'd like to have an ability to deny some cmake features in such external untrusted insertions. 1. Disable any COMMAND invokations (in exec[ute]_command, add_custom_command, add_custom_target etc.) 2. Disable any external_project downloads etc. 3. Disable specific cmake commands (e.g. file(WRITE ...) 4. add more potentially unsecure features here Potential vulnerabilities are (according to previous list): 1. Invocation of 'rm -rf' command. The dir can be / in sudo or ~ in user mode. 2. Downloading of unwanted scripts. Also applicable to 1). 3. Rewriting important system files (like /boot/kernel.image..., /lib/libc... etc) How it can be done? 1. Set a new policy? 2,3. Add new cmake command: enable_command(cmd, {On|Off}) Example: ... # trusted code above set_policy(disable COMMAND) # pseudo enable_command(file, Off) # untrusted ON execute_command( COMMAND wget http://.../evil_script.sh COMMAND ./evil_script.sh ) # will throw an error - COMMAND is denied file(WRITE /boot/vmlinuz-4.4.0-34-generic "") # will throw an error - file is banned # untrusted OFF enable_command(file, On) set_policy(enable COMMAND) # pseudo # trusted code below ... What do you think? Is it possible in the latest CMake? Or can it be added in the future? -- Egor Pugin From tobias.hunger at gmail.com Sat Aug 20 17:25:53 2016 From: tobias.hunger at gmail.com (Tobias Hunger) Date: Sat, 20 Aug 2016 23:25:53 +0200 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: Hi Egor, Am 20.08.2016 13:48 schrieb "Egor Pugin" : > > Hi, > > I'm working on a package manager based on cmake. > And some cmake instructions are downloaded with user packages. > I'd like to have an ability to deny some cmake features in such > external untrusted insertions. I am no CMake expert, but you are talking about securing a program that is meant to take arbitrary input and run user-defined commands on that to produce possibly executable output. I do not see any safe subset of CMake commands that is still able to do anything useful. I can see a way for "insertions" to be useful, that does not involve them changing the configuration (e.g. for a cross compiler), involve running some 3rd party program (e.g. to add support for a new documentation system, parser generator or whatnot), or the production of build artifacts (e.g. build some library for the developer to use). *All* of these are inheritently unsafe. Configuration change: Change the C compiler to rm and pass force -rf -- / as flags. 3rd party program: Run rm -rf / when some certain input file is seen. Build artifacts: Put running rm -rf / into the binary/library so that this is run during normal development workflow. I would try to run my package manager in an environment where running rm -rf is harmless to the overall system health. Virtual machines or containers spring to mind there. Not sure that is feasible. Or come up with insertions signing, etc. so that users can at least know they got what was uploaded and know whom to blame when their systems get wiped. Beat Regards, Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From egor.pugin at gmail.com Sat Aug 20 17:42:13 2016 From: egor.pugin at gmail.com (Egor Pugin) Date: Sun, 21 Aug 2016 00:42:13 +0300 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: Hi Tobias, You are right at many points. It's hard to really secure the system from build system/build artifacts/3rd party apps. But step by step it's possible to decrease number of potential sources of vulnerabilities. Described cmake features would be very helpful for this. Right now I'm just investigating possible security improvements and this is only 'nice-to-have' feature. On 21 August 2016 at 00:25, Tobias Hunger wrote: > Hi Egor, > > Am 20.08.2016 13:48 schrieb "Egor Pugin" : >> >> Hi, >> >> I'm working on a package manager based on cmake. >> And some cmake instructions are downloaded with user packages. >> I'd like to have an ability to deny some cmake features in such >> external untrusted insertions. > > I am no CMake expert, but you are talking about securing a program that is > meant to take arbitrary input and run user-defined commands on that to > produce possibly executable output. > > I do not see any safe subset of CMake commands that is still able to do > anything useful. > > I can see a way for "insertions" to be useful, that does not involve them > changing the configuration (e.g. for a cross compiler), involve running some > 3rd party program (e.g. to add support for a new documentation system, > parser generator or whatnot), or the production of build artifacts (e.g. > build some library for the developer to use). > > *All* of these are inheritently unsafe. > > Configuration change: Change the C compiler to rm and pass force -rf -- / as > flags. > > 3rd party program: Run rm -rf / when some certain input file is seen. > > Build artifacts: Put running rm -rf / into the binary/library so that this > is run during normal development workflow. > > I would try to run my package manager in an environment where running rm -rf > is harmless to the overall system health. Virtual machines or containers > spring to mind there. Not sure that is feasible. > > Or come up with insertions signing, etc. so that users can at least know > they got what was uploaded and know whom to blame when their systems get > wiped. > > Beat Regards, > Tobias -- Egor Pugin From rleigh at codelibre.net Sat Aug 20 18:06:38 2016 From: rleigh at codelibre.net (Roger Leigh) Date: Sat, 20 Aug 2016 23:06:38 +0100 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: <49dffe2f-c82a-963b-5487-3f0c603d20ad@codelibre.net> CMake is not and can never be a replacement for a package manager. They serve entirely different roles. While you can extend CMake to give it the ability to build and install all sorts stuff, that does not make it a package manager. It's a convenient way to build a complex set of interdependent packages, but that's all it is. I think trying to make it do more than that is dangerous; it's currently got a fairly clearly defined scope, and there are numerous established tools to take care of packaging. Basically, I do worry about the potential for scope creep here. Speaking from my own experience as one of the main authors of the Debian build tools (sbuild, schroot): - every package is built individually in a secure sandbox which is created before the build and deleted after it. I wrote a tool specifically for this purpose (schroot)--predating docker, openstack and all the current hip stuff. You should probably use a more up to date technology yourself; the main point I wanted to make was the isolation of the filesystem, network, user etc. for the build step; we run the build step without any functional networking other than the loopback to ensure every package cleanly builds as a self-contained entity - a package might be using CMake for its build, inside the sandbox, but CMake has no involvement at a higher level This means that any malicious or accidental privilege escalation, file deletion or whatever is contained and will not cause problems. And if there is anything wrong, we blow the whole lot away as a matter of course anyway. - when the packaged binary is installed, there is no code execution outside the (optional) maintainer scripts; everything else is just a static set of files which get installed. This is completely decoupled from CMake, or whatever build system was used for the build step. When it comes to CMake building using a super-build and/or running find_package and executing third-party Config scripts, other CMake modules etc. this is clearly unsafe. There's clearly scope for unbounded code execution by this code. While these might be required for the build step, there's no need for them to have any influence or involvement for the package creation step and subsequent installation/upgrade/removal by the package manager. Regards, Roger On 20/08/16 22:42, Egor Pugin wrote: > Hi Tobias, > > You are right at many points. It's hard to really secure the system > from build system/build artifacts/3rd party apps. > But step by step it's possible to decrease number of potential sources > of vulnerabilities. > Described cmake features would be very helpful for this. > Right now I'm just investigating possible security improvements and > this is only 'nice-to-have' feature. > > On 21 August 2016 at 00:25, Tobias Hunger wrote: >> Hi Egor, >> >> Am 20.08.2016 13:48 schrieb "Egor Pugin" : >>> >>> Hi, >>> >>> I'm working on a package manager based on cmake. >>> And some cmake instructions are downloaded with user packages. >>> I'd like to have an ability to deny some cmake features in such >>> external untrusted insertions. >> >> I am no CMake expert, but you are talking about securing a program that is >> meant to take arbitrary input and run user-defined commands on that to >> produce possibly executable output. >> >> I do not see any safe subset of CMake commands that is still able to do >> anything useful. >> >> I can see a way for "insertions" to be useful, that does not involve them >> changing the configuration (e.g. for a cross compiler), involve running some >> 3rd party program (e.g. to add support for a new documentation system, >> parser generator or whatnot), or the production of build artifacts (e.g. >> build some library for the developer to use). >> >> *All* of these are inheritently unsafe. >> >> Configuration change: Change the C compiler to rm and pass force -rf -- / as >> flags. >> >> 3rd party program: Run rm -rf / when some certain input file is seen. >> >> Build artifacts: Put running rm -rf / into the binary/library so that this >> is run during normal development workflow. >> >> I would try to run my package manager in an environment where running rm -rf >> is harmless to the overall system health. Virtual machines or containers >> spring to mind there. Not sure that is feasible. >> >> Or come up with insertions signing, etc. so that users can at least know >> they got what was uploaded and know whom to blame when their systems get >> wiped. >> >> Beat Regards, >> Tobias > > > From tobias.hunger at gmail.com Sun Aug 21 04:24:53 2016 From: tobias.hunger at gmail.com (Tobias Hunger) Date: Sun, 21 Aug 2016 10:24:53 +0200 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: Hi Egor, Am 20.08.2016 23:42 schrieb "Egor Pugin" : > You are right at many points. It's hard to really secure the system > from build system/build artifacts/3rd party apps. > But step by step it's possible to decrease number of potential sources > of vulnerabilities. Retrofitting security into a product that was designed without any consideration for security from the start is going to be hard. I do not know what you want to do, so I am not going to claim it is impossible:-) > Described cmake features would be very helpful for this. > Right now I'm just investigating possible security improvements and > this is only 'nice-to-have' feature. What are the attack scenarios you want to defend against? What should not be possible in your system that currently is in CMake? How do these steps help in securing against those attacks? As a user of CMake I am very much opposed to disabling functionality based on the context: That forces me to keep more state in my head when reading CMakeLists.txt files. CMake does way too much in a way to obscure syntax already! Best regards, Tobias > On 21 August 2016 at 00:25, Tobias Hunger wrote: > > Hi Egor, > > > > Am 20.08.2016 13:48 schrieb "Egor Pugin" : > >> > >> Hi, > >> > >> I'm working on a package manager based on cmake. > >> And some cmake instructions are downloaded with user packages. > >> I'd like to have an ability to deny some cmake features in such > >> external untrusted insertions. > > > > I am no CMake expert, but you are talking about securing a program that is > > meant to take arbitrary input and run user-defined commands on that to > > produce possibly executable output. > > > > I do not see any safe subset of CMake commands that is still able to do > > anything useful. > > > > I can see a way for "insertions" to be useful, that does not involve them > > changing the configuration (e.g. for a cross compiler), involve running some > > 3rd party program (e.g. to add support for a new documentation system, > > parser generator or whatnot), or the production of build artifacts (e.g. > > build some library for the developer to use). > > > > *All* of these are inheritently unsafe. > > > > Configuration change: Change the C compiler to rm and pass force -rf -- / as > > flags. > > > > 3rd party program: Run rm -rf / when some certain input file is seen. > > > > Build artifacts: Put running rm -rf / into the binary/library so that this > > is run during normal development workflow. > > > > I would try to run my package manager in an environment where running rm -rf > > is harmless to the overall system health. Virtual machines or containers > > spring to mind there. Not sure that is feasible. > > > > Or come up with insertions signing, etc. so that users can at least know > > they got what was uploaded and know whom to blame when their systems get > > wiped. > > > > Beat Regards, > > Tobias > > > > -- > Egor Pugin -------------- next part -------------- An HTML attachment was scrubbed... URL: From egor.pugin at gmail.com Sun Aug 21 06:34:41 2016 From: egor.pugin at gmail.com (Egor Pugin) Date: Sun, 21 Aug 2016 13:34:41 +0300 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: > What are the attack scenarios you want to defend against? What should not be possible in your system that currently is in CMake? At least downloading or executing bad scripts and commands. > That forces me to keep more state in my head when reading CMakeLists.txt files. CMake files are generated in my system. That's what I mean when I said 'based on CMake'. It's like compiler compiler like yacc, bison, lex, flex. They are producing output not for human readers, but for computer parsers. And that's why generated code is safe and insertions from users are not. Also in the most cases there's no any insertions at all, so it's rare case. On 21 August 2016 at 11:24, Tobias Hunger wrote: > Hi Egor, > > Am 20.08.2016 23:42 schrieb "Egor Pugin" : >> You are right at many points. It's hard to really secure the system >> from build system/build artifacts/3rd party apps. >> But step by step it's possible to decrease number of potential sources >> of vulnerabilities. > > Retrofitting security into a product that was designed without any > consideration for security from the start is going to be hard. > > I do not know what you want to do, so I am not going to claim it is > impossible:-) > >> Described cmake features would be very helpful for this. >> Right now I'm just investigating possible security improvements and >> this is only 'nice-to-have' feature. > > What are the attack scenarios you want to defend against? What should not be > possible in your system that currently is in CMake? How do these steps help > in securing against those attacks? > > As a user of CMake I am very much opposed to disabling functionality based > on the context: That forces me to keep more state in my head when reading > CMakeLists.txt files. CMake does way too much in a way to obscure syntax > already! > > Best regards, > Tobias > >> On 21 August 2016 at 00:25, Tobias Hunger wrote: >> > Hi Egor, >> > >> > Am 20.08.2016 13:48 schrieb "Egor Pugin" : >> >> >> >> Hi, >> >> >> >> I'm working on a package manager based on cmake. >> >> And some cmake instructions are downloaded with user packages. >> >> I'd like to have an ability to deny some cmake features in such >> >> external untrusted insertions. >> > >> > I am no CMake expert, but you are talking about securing a program that >> > is >> > meant to take arbitrary input and run user-defined commands on that to >> > produce possibly executable output. >> > >> > I do not see any safe subset of CMake commands that is still able to do >> > anything useful. >> > >> > I can see a way for "insertions" to be useful, that does not involve >> > them >> > changing the configuration (e.g. for a cross compiler), involve running >> > some >> > 3rd party program (e.g. to add support for a new documentation system, >> > parser generator or whatnot), or the production of build artifacts (e.g. >> > build some library for the developer to use). >> > >> > *All* of these are inheritently unsafe. >> > >> > Configuration change: Change the C compiler to rm and pass force -rf -- >> > / as >> > flags. >> > >> > 3rd party program: Run rm -rf / when some certain input file is seen. >> > >> > Build artifacts: Put running rm -rf / into the binary/library so that >> > this >> > is run during normal development workflow. >> > >> > I would try to run my package manager in an environment where running rm >> > -rf >> > is harmless to the overall system health. Virtual machines or containers >> > spring to mind there. Not sure that is feasible. >> > >> > Or come up with insertions signing, etc. so that users can at least know >> > they got what was uploaded and know whom to blame when their systems get >> > wiped. >> > >> > Beat Regards, >> > Tobias >> >> >> >> -- >> Egor Pugin -- Egor Pugin From tobias.hunger at gmail.com Sun Aug 21 13:43:17 2016 From: tobias.hunger at gmail.com (Tobias Hunger) Date: Sun, 21 Aug 2016 19:43:17 +0200 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: Hi Egor, Am 21.08.2016 12:34 schrieb "Egor Pugin" : > > > What are the attack scenarios you want to defend against? What should not be possible in your system that currently is in CMake? > > At least downloading or executing bad scripts and commands. What is the attack you want to stop? What are bad scripts and commands in this context? CMake runs lots of commands all the time. Most can be changed by a user, many are changed by the generator based on environment and whatnot. Any of these may be bad commands -- based on configuration. Downloading can be done using internal commands or by running e.g. wget or curl, both of which are pretty widely available on developer machines. > > That forces me to keep more state in my head when reading CMakeLists.txt files. > > CMake files are generated in my system. That's what I mean when I said > 'based on CMake'. Sure. But if CMake gets a secure mode for your generator and if that is merged upstream, then I need to know about that when reading or writing CMakeLists.txt. > It's like compiler compiler like yacc, bison, lex, flex. They are > producing output not for human readers, but for computer parsers. > And that's why generated code is safe and insertions from users are not. Generated code is safe only as long as you very tightly control the environment CMake runs in. > Also in the most cases there's no any insertions at all, so it's rare case. I'm sure you know what you are doing:) Best regards, Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From egor.pugin at gmail.com Sun Aug 21 14:02:57 2016 From: egor.pugin at gmail.com (Egor Pugin) Date: Sun, 21 Aug 2016 21:02:57 +0300 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: > What is the attack you want to stop? What are bad scripts and commands in this context? I wrote them in the first message. For example, - any cmake commands that use COMMAND keyword (execute_process(COMMAND ...), add_custom_{command|target}(...) etc. This will deny any user scripts, programs (wget, curl, rm, ...). - download commands (CMake's builtin curl) - they can fill the drives with trash. > CMake runs lots of commands all the time. Most can be changed by a user, many are changed by the generator based on environment and whatnot. Any of these may be bad commands -- based on configuration. Yes, and it should deny only stuff above in small CMakeLists.txt part that will be protected with some other commands or policies. > But if CMake gets a secure mode for your generator and if that is merged upstream, then I need to know about that when reading or writing CMakeLists.txt. For the moment I'm just asking about possibility of implementation of these features. Any decision will go from CMake guys, not from me. So, you shouldn't ask me about it. :) > Generated code is safe only as long as you very tightly control the environment CMake runs in. I don't care what is around, what is in user env. This is his responsibility. I'm just worrying for my parts of CMake stuff. On 21 August 2016 at 20:43, Tobias Hunger wrote: > Hi Egor, > > Am 21.08.2016 12:34 schrieb "Egor Pugin" : >> >> > What are the attack scenarios you want to defend against? What should >> > not be possible in your system that currently is in CMake? >> >> At least downloading or executing bad scripts and commands. > > What is the attack you want to stop? What are bad scripts and commands in > this context? > > CMake runs lots of commands all the time. Most can be changed by a user, > many are changed by the generator based on environment and whatnot. Any of > these may be bad commands -- based on configuration. > > Downloading can be done using internal commands or by running e.g. wget or > curl, both of which are pretty widely available on developer machines. > >> > That forces me to keep more state in my head when reading CMakeLists.txt >> > files. >> >> CMake files are generated in my system. That's what I mean when I said >> 'based on CMake'. > > Sure. But if CMake gets a secure mode for your generator and if that is > merged upstream, then I need to know about that when reading or writing > CMakeLists.txt. > >> It's like compiler compiler like yacc, bison, lex, flex. They are >> producing output not for human readers, but for computer parsers. >> And that's why generated code is safe and insertions from users are not. > > Generated code is safe only as long as you very tightly control the > environment CMake runs in. > >> Also in the most cases there's no any insertions at all, so it's rare >> case. > > I'm sure you know what you are doing:) > > Best regards, > Tobias -- Egor Pugin From daniel at pfeifer-mail.de Mon Aug 22 05:09:25 2016 From: daniel at pfeifer-mail.de (Daniel Pfeifer) Date: Mon, 22 Aug 2016 11:09:25 +0200 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: Hi Egor, On Sat, Aug 20, 2016 at 1:48 PM, Egor Pugin wrote: > Hi, > > I'm working on a package manager based on cmake. Please rethink that approach. Your package manager should support CMake. It should not be based on it. Your design will be safer and more flexible. Cheers, Daniel From chuck.atkins at kitware.com Mon Aug 22 13:17:39 2016 From: chuck.atkins at kitware.com (Chuck Atkins) Date: Mon, 22 Aug 2016 13:17:39 -0400 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: Hi Egor, Is this intended to run on Linux? If so, I think you're FAR better off leveraging an existing security framework like SELinux, since it's actually designed from the ground up to enforce these types of controls. You could define a label that you place on the executables run by the package manager and then enforce whatever restrictions and controls you feel are appropriate. This will let you do things like block network access got the specific CMake executable used by the package system. It allows the CMake scripts to leverage all the available language and command features but deny, ant a system level, actions you deem unsafe or harmful. ---------- Chuck Atkins Staff R&D Engineer, Scientific Computing Kitware, Inc. <%28518%29%20881-1183> On Sun, Aug 21, 2016 at 2:02 PM, Egor Pugin wrote: > > What is the attack you want to stop? What are bad scripts and commands > in this context? > > I wrote them in the first message. For example, > - any cmake commands that use COMMAND keyword (execute_process(COMMAND > ...), add_custom_{command|target}(...) etc. This will deny any user > scripts, programs (wget, curl, rm, ...). > - download commands (CMake's builtin curl) - they can fill the drives > with trash. > > > CMake runs lots of commands all the time. Most can be changed by a user, > many are changed by the generator based on environment and whatnot. Any of > these may be bad commands -- based on configuration. > > Yes, and it should deny only stuff above in small CMakeLists.txt part > that will be protected with some other commands or policies. > > > But if CMake gets a secure mode for your generator and if that is merged > upstream, then I need to know about that when reading or writing > CMakeLists.txt. > > For the moment I'm just asking about possibility of implementation of > these features. Any decision will go from CMake guys, not from me. So, > you shouldn't ask me about it. :) > > > Generated code is safe only as long as you very tightly control the > environment CMake runs in. > > I don't care what is around, what is in user env. This is his > responsibility. I'm just worrying for my parts of CMake stuff. > > On 21 August 2016 at 20:43, Tobias Hunger wrote: > > Hi Egor, > > > > Am 21.08.2016 12:34 schrieb "Egor Pugin" : > >> > >> > What are the attack scenarios you want to defend against? What should > >> > not be possible in your system that currently is in CMake? > >> > >> At least downloading or executing bad scripts and commands. > > > > What is the attack you want to stop? What are bad scripts and commands in > > this context? > > > > CMake runs lots of commands all the time. Most can be changed by a user, > > many are changed by the generator based on environment and whatnot. Any > of > > these may be bad commands -- based on configuration. > > > > Downloading can be done using internal commands or by running e.g. wget > or > > curl, both of which are pretty widely available on developer machines. > > > >> > That forces me to keep more state in my head when reading > CMakeLists.txt > >> > files. > >> > >> CMake files are generated in my system. That's what I mean when I said > >> 'based on CMake'. > > > > Sure. But if CMake gets a secure mode for your generator and if that is > > merged upstream, then I need to know about that when reading or writing > > CMakeLists.txt. > > > >> It's like compiler compiler like yacc, bison, lex, flex. They are > >> producing output not for human readers, but for computer parsers. > >> And that's why generated code is safe and insertions from users are not. > > > > Generated code is safe only as long as you very tightly control the > > environment CMake runs in. > > > >> Also in the most cases there's no any insertions at all, so it's rare > >> case. > > > > I'm sure you know what you are doing:) > > > > Best regards, > > Tobias > > > > -- > Egor Pugin > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/opensou > rce/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > -------------- next part -------------- An HTML attachment was scrubbed... URL: From egor.pugin at gmail.com Mon Aug 22 13:52:24 2016 From: egor.pugin at gmail.com (Egor Pugin) Date: Mon, 22 Aug 2016 20:52:24 +0300 Subject: [cmake-developers] Security in CMake In-Reply-To: References: Message-ID: Hi Chuck, > Is this intended to run on Linux? Yes. And thanks for the pointing out to SELinux. I'll add it to my checklist. --- The system is on very early stages now, so its parts are changing rapidly and I'm able to consider different approaches to its subsystems (including security). But I probably confused all of you with the words 'package manager'. It's the package manager only in the narrow sense. It's not trying to be another apt, yum etc. Sorry that I didn't provide much details, but the original topic is very precise and I think I'll return to it a bit later when the functionality of the system will be in more stable state. On 22 August 2016 at 20:17, Chuck Atkins wrote: > Hi Egor, > Is this intended to run on Linux? If so, I think you're FAR better off > leveraging an existing security framework like SELinux, since it's actually > designed from the ground up to enforce these types of controls. You could > define a label that you place on the executables run by the package manager > and then enforce whatever restrictions and controls you feel are > appropriate. This will let you do things like block network access got the > specific CMake executable used by the package system. It allows the CMake > scripts to leverage all the available language and command features but > deny, ant a system level, actions you deem unsafe or harmful. > > ---------- > Chuck Atkins > Staff R&D Engineer, Scientific Computing > Kitware, Inc. > > > On Sun, Aug 21, 2016 at 2:02 PM, Egor Pugin wrote: >> >> > What is the attack you want to stop? What are bad scripts and commands >> > in this context? >> >> I wrote them in the first message. For example, >> - any cmake commands that use COMMAND keyword (execute_process(COMMAND >> ...), add_custom_{command|target}(...) etc. This will deny any user >> scripts, programs (wget, curl, rm, ...). >> - download commands (CMake's builtin curl) - they can fill the drives >> with trash. >> >> > CMake runs lots of commands all the time. Most can be changed by a user, >> > many are changed by the generator based on environment and whatnot. Any of >> > these may be bad commands -- based on configuration. >> >> Yes, and it should deny only stuff above in small CMakeLists.txt part >> that will be protected with some other commands or policies. >> >> > But if CMake gets a secure mode for your generator and if that is merged >> > upstream, then I need to know about that when reading or writing >> > CMakeLists.txt. >> >> For the moment I'm just asking about possibility of implementation of >> these features. Any decision will go from CMake guys, not from me. So, >> you shouldn't ask me about it. :) >> >> > Generated code is safe only as long as you very tightly control the >> > environment CMake runs in. >> >> I don't care what is around, what is in user env. This is his >> responsibility. I'm just worrying for my parts of CMake stuff. >> >> On 21 August 2016 at 20:43, Tobias Hunger wrote: >> > Hi Egor, >> > >> > Am 21.08.2016 12:34 schrieb "Egor Pugin" : >> >> >> >> > What are the attack scenarios you want to defend against? What should >> >> > not be possible in your system that currently is in CMake? >> >> >> >> At least downloading or executing bad scripts and commands. >> > >> > What is the attack you want to stop? What are bad scripts and commands >> > in >> > this context? >> > >> > CMake runs lots of commands all the time. Most can be changed by a user, >> > many are changed by the generator based on environment and whatnot. Any >> > of >> > these may be bad commands -- based on configuration. >> > >> > Downloading can be done using internal commands or by running e.g. wget >> > or >> > curl, both of which are pretty widely available on developer machines. >> > >> >> > That forces me to keep more state in my head when reading >> >> > CMakeLists.txt >> >> > files. >> >> >> >> CMake files are generated in my system. That's what I mean when I said >> >> 'based on CMake'. >> > >> > Sure. But if CMake gets a secure mode for your generator and if that is >> > merged upstream, then I need to know about that when reading or writing >> > CMakeLists.txt. >> > >> >> It's like compiler compiler like yacc, bison, lex, flex. They are >> >> producing output not for human readers, but for computer parsers. >> >> And that's why generated code is safe and insertions from users are >> >> not. >> > >> > Generated code is safe only as long as you very tightly control the >> > environment CMake runs in. >> > >> >> Also in the most cases there's no any insertions at all, so it's rare >> >> case. >> > >> > I'm sure you know what you are doing:) >> > >> > Best regards, >> > Tobias >> >> >> >> -- >> Egor Pugin >> -- >> >> Powered by www.kitware.com >> >> Please keep messages on-topic and check the CMake FAQ at: >> http://www.cmake.org/Wiki/CMake_FAQ >> >> Kitware offers various services to support the CMake community. For more >> information on each offering, please visit: >> >> CMake Support: http://cmake.org/cmake/help/support.html >> CMake Consulting: http://cmake.org/cmake/help/consulting.html >> CMake Training Courses: http://cmake.org/cmake/help/training.html >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Follow this link to subscribe/unsubscribe: >> http://public.kitware.com/mailman/listinfo/cmake-developers > > -- Egor Pugin From craig.scott at crascit.com Mon Aug 22 20:06:01 2016 From: craig.scott at crascit.com (Craig Scott) Date: Tue, 23 Aug 2016 10:06:01 +1000 Subject: [cmake-developers] [CMake] Setup/tear down steps for CTest In-Reply-To: References: Message-ID: Cheeky way to get me more involved in contributing, but okay, I'll bite. ;) Switching discussion to the dev list. So how would you want the feature to work? I'd suggest an initial set of requirements something like the following: - Need to support the ability to define multiple setup and/or tear down tasks. - It should be possible to specify dependencies between setup tasks and between tear down tasks. - Individual tests need to be able to indicate which setup and/or tear down tasks they require, similar to the way DEPENDS is used to specify dependencies between test cases. - When using ctest --rerun-failed, ctest should automatically invoke any setup or tear down tasks required by the test cases that will be re-run. - Setup or tear down tasks which reference executable targets should substitute the actual built executable just like how add_custom_command() does. Some open questions: - Should setup and tear down tasks be defined in pairs, or should they completely independent (this would still require the ability to specify a dependency of a tear down task on a setup task)? - Should the setup and tear down tasks be defined by a new CTest/CMake command or extend an existing mechanism (e.g. add_custom_command())? - If no test case has a dependency on a setup or tear down task, should that task be skipped? Perhaps tasks need to have a flag which indicates whether they always run or only if a test case depends on it. - What terminology to use? Things like GoogleTest use terms like test *fixtures* for this sort of thing. The terms setup and tear down are a bit imprecise and cumbersome, so we would probably need something better than those. - Would it make sense for the ctest command line to support disabling setup and/or tear down steps? I can see some potential scenarios where this may be desirable, but maybe this is getting too ambitious for a starting set of requirements. - What should happen if a setup or tear down task fails? How would failure be detected? How would such failures impact things like a CDash test report, etc.? I think that's probably enough to kick off discussions for now. On Sun, Aug 21, 2016 at 11:41 PM, David Cole wrote: > The best thing to do would be to add the feature to ctest, and > contribute to the CMake community. > > I, too, use the "run this test first" and "that test last" technique, > and set up DEPENDS property values to ensure ordering when all tests > are run in parallel. However, as you noted, this does not work to run > subsets of tests reliably. For me, I am able to live with the partial > solution because the vast majority of my tests can be run > independently anyhow and we usually do run all the tests, but a setup > / teardown step for the whole suite would be a welcome addition to > ctest. > > Looking forward to your patch... :-) > > > David C. > > > On Sat, Aug 20, 2016 at 8:32 PM, Craig Scott > wrote: > > Let's say a project defines a bunch of tests which require setup and tear > > down steps before/after all the tests are run (not each individual test, > I'm > > talking here about one setup before all tests are run and one tear down > > after all tests have finished). While this could be done by a script > driving > > CTest itself, it is less desirable since different platforms may need > > different driver scripts and this seems like something CTest should be > able > > to handle itself (if the setup/tear down steps use parts of the build, > that > > only strengthens the case to have them handled by CMake/CTest directly). > > > > It is possible to abuse the DEPENDS test property and define setup and > tear > > down "tests" which are not really tests but which perform the necessary > > steps. While this mostly works, it is not ideal and in particular it > doesn't > > work with CTest's --rerun-failed option. I'm wondering if there's > currently > > a better way of telling CMake/CTest about a setup step which must be run > > before some particular set of test cases and a tear down step after they > are > > all done. The tear down step needs to be performed regardless of whether > any > > of the real test cases pass or fail. > > > > The motivating case is to start up and shutdown a service which a (subset > > of) test cases need running. That service is expensive to set up and > hence > > it isn't feasible to start it up and shut it down for every test case > > individually. > > > > Any ideas? > > > > -- > > Craig Scott > > Melbourne, Australia > > http://crascit.com > > > > -- > > > > Powered by www.kitware.com > > > > Please keep messages on-topic and check the CMake FAQ at: > > http://www.cmake.org/Wiki/CMake_FAQ > > > > Kitware offers various services to support the CMake community. For more > > information on each offering, please visit: > > > > CMake Support: http://cmake.org/cmake/help/support.html > > CMake Consulting: http://cmake.org/cmake/help/consulting.html > > CMake Training Courses: http://cmake.org/cmake/help/training.html > > > > Visit other Kitware open-source projects at > > http://www.kitware.com/opensource/opensource.html > > > > Follow this link to subscribe/unsubscribe: > > http://public.kitware.com/mailman/listinfo/cmake > -- Craig Scott Melbourne, Australia http://crascit.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Hunger at qt.io Mon Aug 22 07:55:43 2016 From: Tobias.Hunger at qt.io (Tobias Hunger) Date: Mon, 22 Aug 2016 11:55:43 +0000 Subject: [cmake-developers] Unix makefile: concurrent build configurations In-Reply-To: References: Message-ID: <1471866942.791.12.camel@qt.io> Hi Steve! On Do, 2016-08-18 at 15:37 -0500, Steve Lorimer wrote: > New to this list, so apologies if it's been discussed before. > > I'd like to know why it's not possible to have both debug and release mode > builds configured at the same time for Unix makefiles? You can have an arbitrary number of configurations at once, all nicely separated from each other in their own build directory. Nobody stops you from having several build directories after all. > I am aware that some generators (eg VisualStudio) do allow this behaviour. > Why can I not have the same behaviour with Unix makefiles? IMHO VisualStudio is rather broken in that regard. Please do not let the insanity pass over to Makefiles! :-) Best Regards, Tobias -- Tobias Hunger,?Senior?Software Engineer?| The Qt Company The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin Gesch?ftsf?hrer: Mika P?lsi, Juha Varelius,?Mika Harjuaho.?Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B From eike at sf-mail.de Tue Aug 23 02:00:09 2016 From: eike at sf-mail.de (Rolf Eike Beer) Date: Tue, 23 Aug 2016 08:00:09 +0200 Subject: [cmake-developers] [CMake] Setup/tear down steps for CTest In-Reply-To: References: Message-ID: <2790033.ITbBjc0OBD@caliban.sf-tec.de> Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: > Cheeky way to get me more involved in contributing, but okay, I'll bite. ;) > Switching discussion to the dev list. > > So how would you want the feature to work? I'd suggest an initial set of > requirements something like the following: > > - Need to support the ability to define multiple setup and/or tear down > tasks. > - It should be possible to specify dependencies between setup tasks and > between tear down tasks. > - Individual tests need to be able to indicate which setup and/or tear > down tasks they require, similar to the way DEPENDS is used to specify > dependencies between test cases. > - When using ctest --rerun-failed, ctest should automatically invoke any > setup or tear down tasks required by the test cases that will be re-run. > - Setup or tear down tasks which reference executable targets should > substitute the actual built executable just like how add_custom_command() > does. -need a way to mark if 2 tests with the same setup/teardown can share those or if they need to run for every of them -the default for each test is "no s/t", which means it can't be run with any of the above in parallel (especially for compatibillity)[1] -need a way to tell if a test doesn't care about those 1) think of a database connector test: the test that will check what happens if no DB is present will fail if the setup step "start DB" was run, but not the teardown > Some open questions: > > - Should setup and tear down tasks be defined in pairs, or should they > completely independent (this would still require the ability to specify a > dependency of a tear down task on a setup task)? The test could be "shutdown daemon" or "delete files" so I would keep them separated. They may be created by the same command, so they could be batched anyway. > - Should the setup and tear down tasks be defined by a new CTest/CMake > command or extend an existing mechanism (e.g. add_custom_command())? Don't clutter existing commands more than needed. If it's something new, then create a new command (they could still share C++ code). If it's basically the same as anything existing at the end then use that. > - If no test case has a dependency on a setup or tear down task, should > that task be skipped? Perhaps tasks need to have a flag which indicates > whether they always run or only if a test case depends on it. Keep backward compatibility. I.e. if I now add a new test with s/t, then every other test should still run (and succeed) as before. > - What terminology to use? Things like GoogleTest use terms like test > *fixtures* for this sort of thing. The terms setup and tear down are a > bit imprecise and cumbersome, so we would probably need something better > than those. > - Would it make sense for the ctest command line to support disabling > setup and/or tear down steps? I can see some potential scenarios where > this may be desirable, but maybe this is getting too ambitious for a > starting set of requirements. IMHO that doesn't make sense. One could think about an option to disable the s/t merging, i.e. that they are really called alone for every test. > - What should happen if a setup or tear down task fails? How would > failure be detected? How would such failures impact things like a CDash > test report, etc.? Then the test fails, just the same as it now does when it can't find the executable. Eike -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From florent.castelli at gmail.com Tue Aug 23 04:57:27 2016 From: florent.castelli at gmail.com (Florent Castelli) Date: Tue, 23 Aug 2016 10:57:27 +0200 Subject: [cmake-developers] Unix makefile: concurrent build configurations In-Reply-To: <1471866942.791.12.camel@qt.io> References: <1471866942.791.12.camel@qt.io> Message-ID: <76b49539-4c2f-03d3-fd5a-b62f5ca5c193@gmail.com> Well, I imagine we could have a shim that will automatically create sub-build directories when you specify a CONFIGURATION=... argument or just starts the build in there. And if a build folder is missing but the configuration is declared, it would run CMake with similar startup arguments. In practice, it would be the same generator, with some extra logic in the main Makefile. /Florent On 22/08/2016 13:55, Tobias Hunger wrote: > Hi Steve! > > On Do, 2016-08-18 at 15:37 -0500, Steve Lorimer wrote: >> New to this list, so apologies if it's been discussed before. >> >> I'd like to know why it's not possible to have both debug and release mode >> builds configured at the same time for Unix makefiles? > You can have an arbitrary number of configurations at once, all nicely separated > from each other in their own build directory. Nobody stops you from having > several build directories after all. > >> I am aware that some generators (eg VisualStudio) do allow this behaviour. >> Why can I not have the same behaviour with Unix makefiles? > IMHO VisualStudio is rather broken in that regard. Please do not let the > insanity pass over to Makefiles! :-) > > Best Regards, > Tobias > From brad.king at kitware.com Tue Aug 23 10:56:48 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 23 Aug 2016 10:56:48 -0400 Subject: [cmake-developers] [PATCH] Tests: Refactor testing of extra generators. In-Reply-To: <1471466903-17145-1-git-send-email-aoe@google.com> References: <1471466903-17145-1-git-send-email-aoe@google.com> Message-ID: On 08/17/2016 04:48 PM, Chaoren Lin via cmake-developers wrote: > From: Chaoren Lin > > Use a loop instead of repeating the same thing multiple times. > --- > Tests/CMakeLists.txt | 77 ++++++++++++++++++++-------------------------------- > 1 file changed, 30 insertions(+), 47 deletions(-) Applied, thanks: Tests: Refactor testing of extra generators. https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=9bd0643a -Brad From brad.king at kitware.com Tue Aug 23 10:56:53 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 23 Aug 2016 10:56:53 -0400 Subject: [cmake-developers] Subject: [PATCH v2] cmFileCommand: Download continuation support In-Reply-To: <320021471542468@web26m.yandex.ru> References: <320021471542468@web26m.yandex.ru> Message-ID: <40312925-d698-6135-f74d-9ef46f9ac6ed@kitware.com> On 08/18/2016 01:47 PM, Titov Denis wrote: > + } else if (*i == "RETRY_COUNT") { > + ++i; > + if (i != args.end()) { > + retryMaxCount = atoi(i->c_str()); > + } else { > + this->SetError("DOWNLOAD missing count for RETRY_COUNT"); Adding tests for these options will be difficult, but we can at least add tests for the error cases. Please look at adding cases to Tests/RunCMake/file/RunCMakeTest.cmake for these. See Tests/RunCMake/README.rst for documentation of how this test infrastructure works. Sorry I didn't notice this during my first review. Thanks, -Brad From steve.lorimer at gmail.com Tue Aug 23 13:11:59 2016 From: steve.lorimer at gmail.com (Steve Lorimer) Date: Tue, 23 Aug 2016 12:11:59 -0500 Subject: [cmake-developers] ctest doesn't respect --output-on-failure if --quiet is also specified Message-ID: Maybe I just don't know how to use ctest properly. I'm trying to suppress ctest's output *except* if there is a test failure. As such, I'm running with the following command: ctest --output-on-failure --quiet According to the man page, --quiet suppresses --verbose, --extra-verbose and --debug. According to the man page, --output-on-failure displays anything outputted by the test program if the test should fail. Is there a way to get the best of both worlds? That is, suppress everything unless there is a failure, only display the failing test's output and ctest's output for the failing test. Here are my various tests cases: *1. ctest with no arguments:* $ ctest Test project /home/steve/src/cmake_test/.build/debug Start 1: foo_test 1/2 Test #1: foo_test ......................... Passed 1.86 sec Start 2: bar_test 2/2 Test #2: bar_test .........................***Failed 1.92 sec 50% tests passed, 1 tests failed out of 2 Total Test time (real) = 3.78 sec The following tests FAILED: 2 - bar_test (Failed) Errors while running CTest *2. ctest with quiet only* $ ctest --quiet *3. ctest with output-on-failure only* $ ctest --output-on-failure Test project /home/steve/src/cmake_test/.build/debug Start 1: foo_test 1/2 Test #1: foo_test ......................... Passed 1.81 sec Start 2: bar_test 2/2 Test #2: bar_test .........................***Failed 1.91 sec Running 1 test case... testing bar /home/steve/src/cmake_test/bar/test/main.cpp(10): fatal error: in "bar_test": critical check bar() == "123 bar" has failed [123 bar foo != 123 bar] *** 1 failure is detected in the test module "Master Test Suite" 50% tests passed, 1 tests failed out of 2 Total Test time (real) = 3.71 sec The following tests FAILED: 2 - bar_test (Failed) Errors while running CTest *4. ctest with both quiet and output-on-failure* $ ctest --output-on-failure --quiet Why no error output? -------------- next part -------------- An HTML attachment was scrubbed... URL: From Don.Olmstead at sony.com Tue Aug 23 13:58:13 2016 From: Don.Olmstead at sony.com (Olmstead, Don) Date: Tue, 23 Aug 2016 17:58:13 +0000 Subject: [cmake-developers] VS Toolchain Support Message-ID: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> Hi, I've been working a bit to get WebKit compiled on Windows using Clang and have been facing some issues with the project files it generates. My setup is VS 2015 Update 3 with the Clang with Microsoft CodeGen (v140_clang_c2). I noticed that the values within the project property pages are incorrect after generation. As an example by default the toolchain in _FLAGS is defining -fms-compatibility. Within the project the default is -fno-ms-compatibility. The value in Configuration Properties -> C/C++ -> Advanced -> Microsoft Compatibility Mode is set to No (-fno-ms-compatibility) not Yes (-fms-compatibility). Under Command Line in C/C++ the All Options contains -fno-ms-compatiblity while in the Additional Options the -fms-extensions is present. What I am wondering is if there is any way to describe to CMake the structure of the project file so that it can properly create the values. On IRC I got pointed at https://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmVS14CLFlagTable.h;h=5812e79e85d89a706306ea3f1eeac93ae0da72de;hb=refs/heads/master so it looks like things might just be hard coded. If so is there any work being done to more accurately create project files for different toolchains? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Wed Aug 24 08:21:36 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 24 Aug 2016 08:21:36 -0400 Subject: [cmake-developers] VS Toolchain Support In-Reply-To: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> References: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> Message-ID: On 08/23/2016 01:58 PM, Olmstead, Don wrote: > is there any work being done to more accurately create project > files for different toolchains? No one is actively working on it AFAIK. The central tracking issue for this is here: https://gitlab.kitware.com/cmake/cmake/issues/16153 Basically one needs to add a bunch more flag tables and then teach the generators to choose a table based on the toolset rather than the generator. The additional tables could be hard-coded in the C++ source. Alternatively one could design a JSON format to look up the tables from some resource directory that comes with CMake. That would allow additional tables to be added without re-compiling. I don't think we should allow projects to provide the files themselves though because we should reserve the right to refactor them at will. -Brad From brad.king at kitware.com Wed Aug 24 14:31:03 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 24 Aug 2016 14:31:03 -0400 Subject: [cmake-developers] VS Toolchain Support In-Reply-To: <92CA7C118E7E1740A32EB4E6361DEFB26B044579@USCULXMSG08.am.sony.com> References: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> <92CA7C118E7E1740A32EB4E6361DEFB26B044579@USCULXMSG08.am.sony.com> Message-ID: On 08/24/2016 02:25 PM, Olmstead, Don wrote: > I would be interested in the JSON format. Do you have any > opinions on what that would look like or would you just > like to see some implementation before proceding? Please propose a sample format. Just take a few entries from one of the existing flag tables and manually convert it to some kind of JSON array of objects to post here for discussion. Thanks, -Brad From Don.Olmstead at sony.com Wed Aug 24 14:25:19 2016 From: Don.Olmstead at sony.com (Olmstead, Don) Date: Wed, 24 Aug 2016 18:25:19 +0000 Subject: [cmake-developers] VS Toolchain Support In-Reply-To: References: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> Message-ID: <92CA7C118E7E1740A32EB4E6361DEFB26B044579@USCULXMSG08.am.sony.com> Thanks for the response Brad! I would be interested in the JSON format. Do you have any opinions on what that would look like or would you just like to see some implementation before proceding? -----Original Message----- From: Brad King [mailto:brad.king at kitware.com] Sent: Wednesday, August 24, 2016 5:22 AM To: Olmstead, Don Cc: cmake-developers at cmake.org Subject: Re: [cmake-developers] VS Toolchain Support On 08/23/2016 01:58 PM, Olmstead, Don wrote: > is there any work being done to more accurately create project files > for different toolchains? No one is actively working on it AFAIK. The central tracking issue for this is here: https://gitlab.kitware.com/cmake/cmake/issues/16153 Basically one needs to add a bunch more flag tables and then teach the generators to choose a table based on the toolset rather than the generator. The additional tables could be hard-coded in the C++ source. Alternatively one could design a JSON format to look up the tables from some resource directory that comes with CMake. That would allow additional tables to be added without re-compiling. I don't think we should allow projects to provide the files themselves though because we should reserve the right to refactor them at will. -Brad From Don.Olmstead at sony.com Wed Aug 24 15:39:09 2016 From: Don.Olmstead at sony.com (Olmstead, Don) Date: Wed, 24 Aug 2016 19:39:09 +0000 Subject: [cmake-developers] VS Toolchain Support In-Reply-To: References: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> <92CA7C118E7E1740A32EB4E6361DEFB26B044579@USCULXMSG08.am.sony.com> Message-ID: <92CA7C118E7E1740A32EB4E6361DEFB26B0445CC@USCULXMSG08.am.sony.com> For a first pass it might be best to just follow what is in cmIDEFlagTable as this would largely be unintrusive and just get the architecture there. That would look like { "library":[ { "name":"ErrorReporting", "command":"ERRORREPORT:PROMPT", "comment":"PromptImmediately", "value":"PromptImmediately" }, { "name":"ErrorReporting", "command":"ERRORREPORT:QUEUE", "comment":"Queue For Next Login", "value":"QueueForNextLogin" } .. ] } Where flags would be optional. Another option would actually group the values together. { "library":{ "ErrorReporting":{ "values":[ { "command":"ERRORREPORT:PROMPT", "comment":"PromptImmediately", "value":"PromptImmediately" }, { "command":"ERRORREPORT:QUEUE", "comment":"Queue For Next Login", "value":"QueueForNextLogin" } ] } } } Again flags would be optional. If there are not multiple values then it would look something like this. { "library":{ "AdditionalLibraryDirectories":{ "command":"LIBPATH:", "comment":"Additional Library Directories", "value":"", "flags":[ "UserValue", "SemicolonAppendable" ] } } } You could also do instead of flags a type. So "type": "StringList" which would then map the flags properly. I'm up for whatever you think would be best. The first is certainly the most verbose but has a direct mapping. The rest end up requiring more of a transformation to get it into the current format. -----Original Message----- From: Brad King [mailto:brad.king at kitware.com] Sent: Wednesday, August 24, 2016 11:31 AM To: Olmstead, Don Cc: cmake-developers at cmake.org Subject: Re: [cmake-developers] VS Toolchain Support On 08/24/2016 02:25 PM, Olmstead, Don wrote: > I would be interested in the JSON format. Do you have any opinions on > what that would look like or would you just like to see some > implementation before proceding? Please propose a sample format. Just take a few entries from one of the existing flag tables and manually convert it to some kind of JSON array of objects to post here for discussion. Thanks, -Brad From Don.Olmstead at sony.com Wed Aug 24 21:35:09 2016 From: Don.Olmstead at sony.com (Olmstead, Don) Date: Thu, 25 Aug 2016 01:35:09 +0000 Subject: [cmake-developers] VS Toolchain Support In-Reply-To: <92CA7C118E7E1740A32EB4E6361DEFB26B0445CC@USCULXMSG08.am.sony.com> References: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> <92CA7C118E7E1740A32EB4E6361DEFB26B044579@USCULXMSG08.am.sony.com> <92CA7C118E7E1740A32EB4E6361DEFB26B0445CC@USCULXMSG08.am.sony.com> Message-ID: <92CA7C118E7E1740A32EB4E6361DEFB26B04489B@USCULXMSG08.am.sony.com> Did a bit of poking around the CMake codebase to see what a patch might look like. Was thinking something along the lines of this. #ifndef cmVisualStudio10ToolsetOptions_h #define cmVisualStudio10ToolsetOptions_h #include "cmStandardIncludes.h" struct cmIDEFlagTable; /** \class cmVisualStudio10ToolsetOptions * \brief Retrieves toolset options for MSBuild. * * cmVisualStudio10ToolsetOptions manages toolsets within MSBuild */ class cmVisualStudio10ToolsetOptions { public: cmIDEFlagTable const* GetClFlagTable(std::string name) const; cmIDEFlagTable const* GetRcFlagTable(std::string name) const; cmIDEFlagTable const* GetLibFlagTable(std::string name) const; cmIDEFlagTable const* GetLinkFlagTable(std::string name) const; cmIDEFlagTable const* GetMasmFlagTable(std::string name) const; }; #endif This particular class would handle the loading of the JSON data and returning it to the client code in the current cmIDEFlagTable setup. One possible addition might be to include the target architecture. It appears that at least within VS 2015 the Toolsets are organized first by architecture (MSBuild\Microsoft.Cpp\v4.0\V140\Platforms). It feels like it should possibly get bolted onto cmGlobalVisualStudio10Generator with theGet*FlagTable methods moved over from cmVisualStudio10TargetGenerator. Also from looking at the various definitions within MSBuild that creating a tool that parses the XML definitions into what's expected by whatever the JSON format ends up looking like wouldn't be too bad. I started a script in Python that would probably do the trick. If there's a preferred language though I can use that assuming something along those lines would be accepted in a patch. When contributing would you prefer a series of patches or just a single large patch? From joubert.sy at gmail.com Thu Aug 25 06:04:46 2016 From: joubert.sy at gmail.com (Sylvain Joubert) Date: Thu, 25 Aug 2016 12:04:46 +0200 Subject: [cmake-developers] [PATCH] Make test driver clang-tidy clear Message-ID: Hi, Please find attach a patch that clean the test driver source regarding clang-tidy checks. Especially, I've replaced the use of strcpy by strncpy. This should avoid users to get clang-tidy warnings in their build from CMake generated code. Regards, Sylvain -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Use-safer-strncpy-instead-of-strcpy.patch Type: text/x-diff Size: 1424 bytes Desc: not available URL: From brad.king at kitware.com Thu Aug 25 09:05:18 2016 From: brad.king at kitware.com (Brad King) Date: Thu, 25 Aug 2016 09:05:18 -0400 Subject: [cmake-developers] VS Toolchain Support In-Reply-To: <92CA7C118E7E1740A32EB4E6361DEFB26B04489B@USCULXMSG08.am.sony.com> References: <92CA7C118E7E1740A32EB4E6361DEFB26B043D35@USCULXMSG08.am.sony.com> <92CA7C118E7E1740A32EB4E6361DEFB26B044579@USCULXMSG08.am.sony.com> <92CA7C118E7E1740A32EB4E6361DEFB26B0445CC@USCULXMSG08.am.sony.com> <92CA7C118E7E1740A32EB4E6361DEFB26B04489B@USCULXMSG08.am.sony.com> Message-ID: <08fb4ddb-ae08-61c8-cdb5-e8aa68f60696@kitware.com> On 08/24/2016 03:39 PM, Olmstead, Don wrote: > For a first pass it might be best to just follow what is in cmIDEFlagTable > as this would largely be unintrusive and just get the architecture there. Yes, let's go with a direct mapping for now. These files don't necessarily have to be pretty. On 08/24/2016 09:35 PM, Olmstead, Don wrote: > class cmVisualStudio10ToolsetOptions > { > public: > cmIDEFlagTable const* GetClFlagTable(std::string name) const; > cmIDEFlagTable const* GetRcFlagTable(std::string name) const; > cmIDEFlagTable const* GetLibFlagTable(std::string name) const; > cmIDEFlagTable const* GetLinkFlagTable(std::string name) const; > cmIDEFlagTable const* GetMasmFlagTable(std::string name) const; > }; > #endif Looks good. One challenge will be the toolset lookup. There are many variations of some toolset names and we need to figure out how to map to a specific JSON file without duplicating them excessively. > One possible addition might be to include the target architecture. It appears that > at least within VS 2015 the Toolsets are organized first by architecture > (MSBuild\Microsoft.Cpp\v4.0\V140\Platforms). Yes, we can add that if needed. > It feels like it should possibly get bolted onto cmGlobalVisualStudio10Generator > with the Get*FlagTable methods moved over from cmVisualStudio10TargetGenerator. Yes. > Also from looking at the various definitions within MSBuild that creating a tool > that parses the XML definitions into what's expected by whatever the JSON format > ends up looking like wouldn't be too bad. I started a script in Python that would > probably do the trick. If there's a preferred language though I can use that > assuming something along those lines would be accepted in a patch. For one-shot conversions any language is fine. Source/cmparseMSBuildXML.py is the script we currently use for this (followed by some hand editing). > When contributing would you prefer a series of patches or just a > single large patch? We prefer series of incremental patches. Thanks, -Brad From ben.boeckel at kitware.com Thu Aug 25 10:08:10 2016 From: ben.boeckel at kitware.com (Ben Boeckel) Date: Thu, 25 Aug 2016 10:08:10 -0400 Subject: [cmake-developers] [CMake] Setup/tear down steps for CTest In-Reply-To: <2790033.ITbBjc0OBD@caliban.sf-tec.de> References: <2790033.ITbBjc0OBD@caliban.sf-tec.de> Message-ID: <20160825140810.GA13534@megas.kitware.com> On Tue, Aug 23, 2016 at 08:00:09 +0200, Rolf Eike Beer wrote: > Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: > > So how would you want the feature to work? I'd suggest an initial set of > > requirements something like the following: > > > > - Need to support the ability to define multiple setup and/or tear down > > tasks. > > - It should be possible to specify dependencies between setup tasks and > > between tear down tasks. > > - Individual tests need to be able to indicate which setup and/or tear > > down tasks they require, similar to the way DEPENDS is used to specify > > dependencies between test cases. > > - When using ctest --rerun-failed, ctest should automatically invoke any > > setup or tear down tasks required by the test cases that will be re-run. > > - Setup or tear down tasks which reference executable targets should > > substitute the actual built executable just like how add_custom_command() > > does. > > -need a way to mark if 2 tests with the same setup/teardown can share those or > if they need to run for every of them Proposal: add_test(NAME setup-foo ...) set_tests_properties(setup-foo PROPERTIES SETUP_GROUP foo SETUP_STEP SETUP_PER_TEST) # Also SETUP_ONCE. add_test(NAME use-foo ...) set_tests_properties(use-foo PROPERTIES SETUP_GROUP foo) # implicit depends on all SETUP_GROUP foo / SETUP_STEP SETUP_* tests. add_test(NAME use-foo2 ...) set_tests_properties(use-foo2 PROPERTIES SETUP_GROUP foo) add_test(NAME teardown-foo2 ...) set_tests_properties(teardown-foo2 PROPERTIES SETUP_GROUP foo SETUP_STEP TEARDOWN) # implicit depends on all non-TEARDOWN steps Multiple setup/teardown steps could be done with DEPENDS between them. > -the default for each test is "no s/t", which means it can't be run with any > of the above in parallel (especially for compatibillity)[1] > -need a way to tell if a test doesn't care about those Making RESOURCE_LOCK a rwlock rather than a mutex might make sense here. SETUP_STEP bits have a RESOURCE_LOCK_WRITE group_${group}, otherwise it is RESOURCE_LOCK_READ group_${group}. > 1) think of a database connector test: the test that will check what happens > if no DB is present will fail if the setup step "start DB" was run, but not > the teardown RESOURCE_LOCK on that group_${group} can fix that I think. > > Some open questions: I agree with what Eike said. --Ben From brad.king at kitware.com Thu Aug 25 11:00:25 2016 From: brad.king at kitware.com (Brad King) Date: Thu, 25 Aug 2016 11:00:25 -0400 Subject: [cmake-developers] [PATCH] Make test driver clang-tidy clear In-Reply-To: References: Message-ID: <410c9085-eeb5-e5cf-3dd4-9c60183d07f5@kitware.com> On 08/25/2016 06:04 AM, Sylvain Joubert wrote: > Please find attach a patch that clean the test driver source regarding clang-tidy checks. Thanks, applied: create_test_sourcelist: Use safer strncpy instead of strcpy https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=cd344e3a -Brad From ervin at kde.org Thu Aug 25 14:54:18 2016 From: ervin at kde.org (Kevin Ottens) Date: Thu, 25 Aug 2016 20:54:18 +0200 Subject: [cmake-developers] [PATCH] QtCreator integration: C++ includes must come first Message-ID: <4189070.ZWk9KdmZe8@wintermute> Hello, When using the Clang Code Model in QtCreator, it turned out that having the C system include dirs can make it report false positives for most uses on the STL. This is due to the order the Clang Code Model looks at the include directories and some C includes in /usr/include could be incompatible with the used STL if found first. For further details on how this is currently processed within QtCreator and how the issue shows up on my computer, you can take a look at this issue: https://bugreports.qt.io/browse/QTCREATORBUG-16432 It got a previous fix earlier, but it looks like I am in a loophole which requires slight adjustment of the previous commit by Tobias Hunger. I've been successfully using the attached patch for the last two weeks now. Obviously I'd like to see it merged upstream instead of having this locally only. It looks trivial enough to me, but please let me know if something needs to be changed with it which I didn't spot. Regards. -- K?vin Ottens, http://ervin.ipsquad.net KDAB - proud supporter of KDE, http://www.kdab.com -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-QtCreator-integration-C-includes-must-come-first.patch Type: text/x-patch Size: 1689 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: This is a digitally signed message part. URL: From brad.king at kitware.com Fri Aug 26 09:59:16 2016 From: brad.king at kitware.com (Brad King) Date: Fri, 26 Aug 2016 09:59:16 -0400 Subject: [cmake-developers] [PATCH] QtCreator integration: C++ includes must come first In-Reply-To: <4189070.ZWk9KdmZe8@wintermute> References: <4189070.ZWk9KdmZe8@wintermute> Message-ID: On 08/25/2016 02:54 PM, Kevin Ottens wrote: > When using the Clang Code Model in QtCreator, it turned out that having > the C system include dirs can make it report false positives for most > uses on the STL. This is due to the order the Clang Code Model looks at > the include directories and some C includes in /usr/include could be > incompatible with the used STL if found first. Thanks, applied: CodeBlocks: List C++ includes first https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=38995d19 -Brad From tobias.hunger at gmail.com Fri Aug 26 10:49:18 2016 From: tobias.hunger at gmail.com (Tobias Hunger) Date: Fri, 26 Aug 2016 16:49:18 +0200 Subject: [cmake-developers] [PATCH] QtCreator integration: C++ includes must come first In-Reply-To: References: <4189070.ZWk9KdmZe8@wintermute> Message-ID: Hello CMake developers, CMake used to break C++ projects in its CodeBlocks generator. Now it probably breaks C projects. I think the approach is wrong: C and C++ includes should never get concatenated. Please consider to report compiler flags/include paths/defines by language. You might want to keep the concatenated information for backwards compatibility. CMake 3.6 changed the include paths in the CodeBlocks generator to not be in random order -- which might actually have caused the issue Kevin is seeing. The IDEs using CodeBlocks generator seem to be pretty robust at the Include paths front... considering there were no complaints for *years* about the broken include paths ordering. So maybe it is safe to not bother about backward compatibility here. Best Regards, Tobias On Fri, Aug 26, 2016 at 3:59 PM, Brad King wrote: > On 08/25/2016 02:54 PM, Kevin Ottens wrote: >> When using the Clang Code Model in QtCreator, it turned out that having >> the C system include dirs can make it report false positives for most >> uses on the STL. This is due to the order the Clang Code Model looks at >> the include directories and some C includes in /usr/include could be >> incompatible with the used STL if found first. > > Thanks, applied: > > CodeBlocks: List C++ includes first > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=38995d19 > > -Brad From brad.king at kitware.com Fri Aug 26 11:02:18 2016 From: brad.king at kitware.com (Brad King) Date: Fri, 26 Aug 2016 11:02:18 -0400 Subject: [cmake-developers] [PATCH] QtCreator integration: C++ includes must come first In-Reply-To: References: <4189070.ZWk9KdmZe8@wintermute> Message-ID: On 08/26/2016 10:49 AM, Tobias Hunger wrote: > CMake used to break C++ projects in its CodeBlocks generator. Now it > probably breaks C projects. > > I think the approach is wrong: C and C++ includes should never get > concatenated. Please consider to report compiler flags/include > paths/defines by language. You might want to keep the concatenated > information for backwards compatibility. We have no maintainer for the CodeBlocks generator. I welcome anyone with some expertise in it to step forward. -Brad From Robert.Goulet at autodesk.com Fri Aug 26 12:43:38 2016 From: Robert.Goulet at autodesk.com (Robert Goulet) Date: Fri, 26 Aug 2016 16:43:38 +0000 Subject: [cmake-developers] VS custom commands with VS variable containing space in executable path? Message-ID: <960a7eb610494c5b887e3d670558ca12@CO1PR79MB010.MGDADSK.autodesk.com> Hello, I am trying to use a custom command to generate test certificate for a Universal Windows Platform app, and I can't find how to properly quote the executable paths from CMake. Here is how we add the custom command: set(WIN64UWP_CERT_NAME "${PROJECT_NAME}_TemporaryKey") add_custom_command( OUTPUT "${PROJECT_BINARY_DIR}/${WIN64UWP_CERT_NAME}.pfx" COMMAND "$(WindowsSdkDir)/bin/x64/makecert.exe" -cy end -eku 1.3.6.1.5.5.7.3.3 -r -n "CN=${PRODUCT_COMPANY}" -sv ${RESOURCES_DIR}/${WIN64UWP_CERT_NAME}.pvk ${WIN64UWP_CERT_NAME}.cer COMMAND "$(WindowsSdkDir)/bin/x64/PVK2PFX.exe" -pvk ${RESOURCES_DIR}/${WIN64UWP_CERT_NAME}.pvk -spc ${WIN64UWP_CERT_NAME}.cer -pfx ${WIN64UWP_CERT_NAME}.pfx -f WORKING_DIRECTORY ${PROJECT_BINARY_DIR} COMMENT "Generating Test Certificate..." ) Because $(WindowsSdkDir) points within C:\Program Files\... Visual Studio is unable to execute these commands since the .vcxproj does not contain quotes around the executable paths. First I tried using \" around the command, but CMake refuses to escape quotes on the command with error "COMMAND may not contain literal quotes". Then I tried XML escaped character " but then CMake translate the ; as it if was a different element of an array surrounding both sides with quotes. Then I tried %22 but CMake outputs % as %% in the vcxproj file. So how do we setup a custom command that uses a Visual Studio variable that requires to be quoted to preserve spaces? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Fri Aug 26 13:30:16 2016 From: brad.king at kitware.com (Brad King) Date: Fri, 26 Aug 2016 13:30:16 -0400 Subject: [cmake-developers] VS custom commands with VS variable containing space in executable path? In-Reply-To: <960a7eb610494c5b887e3d670558ca12@CO1PR79MB010.MGDADSK.autodesk.com> References: <960a7eb610494c5b887e3d670558ca12@CO1PR79MB010.MGDADSK.autodesk.com> Message-ID: <5c829c95-4c1d-21c2-b625-cdd970d77464@kitware.com> On 08/26/2016 12:43 PM, Robert Goulet wrote: > COMMAND "$(WindowsSdkDir)/bin/x64/makecert.exe" > > So how do we setup a custom command that uses a Visual Studio variable > that requires to be quoted to preserve spaces? I don't think it is supported. You could try adding a wrapper around the call so that it appears as an argument instead of in the command. COMMAND ${CMAKE_COMMAND} -E env "$(WindowsSdkDir)/bin/x64/makecert.exe" ... OTOH you're trying to pierce CMake's abstract model of a command (which is why it's not supported). Instead you could find_program the tools you need and reference them by absolute path. -Brad From Robert.Goulet at autodesk.com Fri Aug 26 13:57:03 2016 From: Robert.Goulet at autodesk.com (Robert Goulet) Date: Fri, 26 Aug 2016 17:57:03 +0000 Subject: [cmake-developers] VS custom commands with VS variable containing space in executable path? In-Reply-To: <5c829c95-4c1d-21c2-b625-cdd970d77464@kitware.com> References: <960a7eb610494c5b887e3d670558ca12@CO1PR79MB010.MGDADSK.autodesk.com> <5c829c95-4c1d-21c2-b625-cdd970d77464@kitware.com> Message-ID: <3d98237886804169b0929657146bbeef@CO1PR79MB010.MGDADSK.autodesk.com> Thanks, this worked: COMMAND ${CMAKE_COMMAND} -E env "\"$(WindowsSdkDir)/bin/x64/makecert.exe\"" ... I still believe that what I was trying to do was perfectly valid in the sense that it is using a VS environment variable $(WindowsSdkDir) that is not defined in the system environment variable, thus it exist only when command is executed from within VS. Normally we do use find_program(...) but this doesn't apply for this case because VS environment variables such as $(WindowsSdkDir) won't resolve outside of VS. Thanks! -----Original Message----- From: Brad King [mailto:brad.king at kitware.com] Sent: Friday, August 26, 2016 1:30 PM To: Robert Goulet Cc: cmake-developers at cmake.org Subject: Re: [cmake-developers] VS custom commands with VS variable containing space in executable path? On 08/26/2016 12:43 PM, Robert Goulet wrote: > COMMAND "$(WindowsSdkDir)/bin/x64/makecert.exe" > > So how do we setup a custom command that uses a Visual Studio variable > that requires to be quoted to preserve spaces? I don't think it is supported. You could try adding a wrapper around the call so that it appears as an argument instead of in the command. COMMAND ${CMAKE_COMMAND} -E env "$(WindowsSdkDir)/bin/x64/makecert.exe" ... OTOH you're trying to pierce CMake's abstract model of a command (which is why it's not supported). Instead you could find_program the tools you need and reference them by absolute path. -Brad From seblist at xwmw.org Sun Aug 28 10:00:02 2016 From: seblist at xwmw.org (Sebastian Holtermann) Date: Sun, 28 Aug 2016 16:00:02 +0200 Subject: [cmake-developers] QtAutogen: Use Base32 instead of Base64 for file names Message-ID: Hello again. I think there are some issues when using Base64 checksum for filenames. - They look weird/ugly with random cases an '_' and '-' characters mixed in - On cases insensitive filesystems mixed case names may cause problems (e.g. undetected name collisions) Base32 is a better choice IMO because it is single case and does not generate disruptive characters like '_', '-' or '/'. Here are two patches that replace Base64 with Base32. Please review. Kind regards, -Sebastian -------------- next part -------------- A non-text attachment was scrubbed... Name: cmake_3-6-1_autogen_patches_base32_v001.tar.gz Type: application/gzip Size: 3187 bytes Desc: not available URL: From amubtdx at outlook.fr Sun Aug 28 11:43:44 2016 From: amubtdx at outlook.fr (Alexis Murzeau) Date: Sun, 28 Aug 2016 15:43:44 +0000 Subject: [cmake-developers] [PATCH] FindOpenSSL: Fix detection of OpenSSL 1.1 w/ MSVC Message-ID: Since OpenSSL 1.1.0, Windows binaries are libcrypto and libssl instead of the old names libeay32 and ssleay32. When using MSVC, FindOpenSSL was searching for the old lib names only so this add the new names to be able to find OpenSSL 1.1.0 libraries. For example, the files in lib directory of OpenSSL 1.1.0 Win64 : - libcrypto.lib - libssl.lib - VC/libcrypto64MD.lib - VC/libcrypto64MDd.lib - VC/libcrypto64MT.lib - VC/libcrypto64MTd.lib - VC/libssl64MD.lib - VC/libssl64MDd.lib - VC/libssl64MT.lib - VC/libssl64MTd.lib 32 bits OpenSSL has the same file with "32" instead of "64" for files in VC directory. MinGW still works and use lib/libcrypto.lib and lib/libssl.lib. --- Modules/FindOpenSSL.cmake | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/Modules/FindOpenSSL.cmake b/Modules/FindOpenSSL.cmake index 10b62ff..ba29a80 100644 --- a/Modules/FindOpenSSL.cmake +++ b/Modules/FindOpenSSL.cmake @@ -133,6 +133,13 @@ if(WIN32 AND NOT CYGWIN) set(_OPENSSL_MSVC_RT_MODE "MD") endif () + # Since OpenSSL 1.1, lib names are like libcrypto32MTd.lib and libssl32MTd.lib + if( "${CMAKE_SIZEOF_VOID_P}" STREQUAL "8" ) + set(_OPENSSL_MSVC_ARCH_SUFFIX "64") + else() + set(_OPENSSL_MSVC_ARCH_SUFFIX "32") + endif() + if(OPENSSL_USE_STATIC_LIBS) set(_OPENSSL_PATH_SUFFIXES "lib" @@ -150,7 +157,9 @@ if(WIN32 AND NOT CYGWIN) find_library(LIB_EAY_DEBUG NAMES libeay32${_OPENSSL_MSVC_RT_MODE}d + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d libeay32d + libcryptod NAMES_PER_DIR ${_OPENSSL_ROOT_HINTS_AND_PATHS} PATH_SUFFIXES @@ -160,7 +169,9 @@ if(WIN32 AND NOT CYGWIN) find_library(LIB_EAY_RELEASE NAMES libeay32${_OPENSSL_MSVC_RT_MODE} + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} libeay32 + libcrypto NAMES_PER_DIR ${_OPENSSL_ROOT_HINTS_AND_PATHS} PATH_SUFFIXES @@ -170,7 +181,9 @@ if(WIN32 AND NOT CYGWIN) find_library(SSL_EAY_DEBUG NAMES ssleay32${_OPENSSL_MSVC_RT_MODE}d + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d ssleay32d + libssld NAMES_PER_DIR ${_OPENSSL_ROOT_HINTS_AND_PATHS} PATH_SUFFIXES @@ -180,8 +193,10 @@ if(WIN32 AND NOT CYGWIN) find_library(SSL_EAY_RELEASE NAMES ssleay32${_OPENSSL_MSVC_RT_MODE} + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} ssleay32 ssl + libssl NAMES_PER_DIR ${_OPENSSL_ROOT_HINTS_AND_PATHS} PATH_SUFFIXES -- 2.7.4.windows.1 From rleigh at codelibre.net Sun Aug 28 13:51:25 2016 From: rleigh at codelibre.net (Roger Leigh) Date: Sun, 28 Aug 2016 18:51:25 +0100 Subject: [cmake-developers] [patch] Expose GNUInstallDirs absolute path algorithm Message-ID: <552e0cb1-0ea8-802f-4710-ac1cd5b51e29@codelibre.net> In several projects I've had a need to compute the "full" absolute path for an installation directory. The logic to do this is in the GNUInstallDirs module, used to compute the CMAKE_INSTALL_FULL_* variables for each of the CMAKE_INSTALL_* cache variables it defines. In order to allow for this logic to be reused, I've separated it out into a macro "set_full_install_dir". This can then be used by any end user to compute their own absolute paths using the same logic. The reason for this is that my projects are creating their own project-specific installation directory cache variables based upon the GNUInstallDirs variables, or sometimes even from scratch. I then need to compute the full path. But I need this macro to be able to do that to match the GNUInstallDirs behaviour. The macro name might need adjusting, but I hope the basic approach is OK in principle. Kind regards, Roger -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-GNUInstallDirs-Add-set_full_install_dir-macro.patch Type: text/x-diff Size: 4123 bytes Desc: not available URL: From craig.scott at crascit.com Sun Aug 28 17:18:44 2016 From: craig.scott at crascit.com (Craig Scott) Date: Mon, 29 Aug 2016 07:18:44 +1000 Subject: [cmake-developers] [PATCH] FindOpenSSL: Fix detection of OpenSSL 1.1 w/ MSVC In-Reply-To: References: Message-ID: In that patch, the old names are listed before the new names, so if both exist on the system, the older version would be found first. Would it be preferable instead to swap the order to give preference to the newer version in such a scenario? On Mon, Aug 29, 2016 at 1:43 AM, Alexis Murzeau wrote: > Since OpenSSL 1.1.0, Windows binaries are libcrypto and libssl instead of > the old names libeay32 and ssleay32. > When using MSVC, FindOpenSSL was searching for the old lib names only so > this add the new names to be able to find OpenSSL 1.1.0 libraries. > > For example, the files in lib directory of OpenSSL 1.1.0 Win64 : > - libcrypto.lib > - libssl.lib > - VC/libcrypto64MD.lib > - VC/libcrypto64MDd.lib > - VC/libcrypto64MT.lib > - VC/libcrypto64MTd.lib > - VC/libssl64MD.lib > - VC/libssl64MDd.lib > - VC/libssl64MT.lib > - VC/libssl64MTd.lib > > 32 bits OpenSSL has the same file with "32" instead of "64" for files in > VC directory. > > MinGW still works and use lib/libcrypto.lib and lib/libssl.lib. > --- > Modules/FindOpenSSL.cmake | 15 +++++++++++++++ > 1 file changed, 15 insertions(+) > > diff --git a/Modules/FindOpenSSL.cmake b/Modules/FindOpenSSL.cmake > index 10b62ff..ba29a80 100644 > --- a/Modules/FindOpenSSL.cmake > +++ b/Modules/FindOpenSSL.cmake > @@ -133,6 +133,13 @@ if(WIN32 AND NOT CYGWIN) > set(_OPENSSL_MSVC_RT_MODE "MD") > endif () > > + # Since OpenSSL 1.1, lib names are like libcrypto32MTd.lib and > libssl32MTd.lib > + if( "${CMAKE_SIZEOF_VOID_P}" STREQUAL "8" ) > + set(_OPENSSL_MSVC_ARCH_SUFFIX "64") > + else() > + set(_OPENSSL_MSVC_ARCH_SUFFIX "32") > + endif() > + > if(OPENSSL_USE_STATIC_LIBS) > set(_OPENSSL_PATH_SUFFIXES > "lib" > @@ -150,7 +157,9 @@ if(WIN32 AND NOT CYGWIN) > find_library(LIB_EAY_DEBUG > NAMES > libeay32${_OPENSSL_MSVC_RT_MODE}d > + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d > libeay32d > + libcryptod > NAMES_PER_DIR > ${_OPENSSL_ROOT_HINTS_AND_PATHS} > PATH_SUFFIXES > @@ -160,7 +169,9 @@ if(WIN32 AND NOT CYGWIN) > find_library(LIB_EAY_RELEASE > NAMES > libeay32${_OPENSSL_MSVC_RT_MODE} > + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} > libeay32 > + libcrypto > NAMES_PER_DIR > ${_OPENSSL_ROOT_HINTS_AND_PATHS} > PATH_SUFFIXES > @@ -170,7 +181,9 @@ if(WIN32 AND NOT CYGWIN) > find_library(SSL_EAY_DEBUG > NAMES > ssleay32${_OPENSSL_MSVC_RT_MODE}d > + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d > ssleay32d > + libssld > NAMES_PER_DIR > ${_OPENSSL_ROOT_HINTS_AND_PATHS} > PATH_SUFFIXES > @@ -180,8 +193,10 @@ if(WIN32 AND NOT CYGWIN) > find_library(SSL_EAY_RELEASE > NAMES > ssleay32${_OPENSSL_MSVC_RT_MODE} > + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} > ssleay32 > ssl > + libssl > NAMES_PER_DIR > ${_OPENSSL_ROOT_HINTS_AND_PATHS} > PATH_SUFFIXES > -- > 2.7.4.windows.1 > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at http://www.kitware.com/ > opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > -- Craig Scott Melbourne, Australia http://crascit.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From amubtdx at outlook.fr Mon Aug 29 14:55:39 2016 From: amubtdx at outlook.fr (Alexis Murzeau) Date: Mon, 29 Aug 2016 18:55:39 +0000 Subject: [cmake-developers] [PATCH] FindOpenSSL: Fix detection of OpenSSL 1.1 w/ MSVC In-Reply-To: References: Message-ID: NAMES_PER_DIR is enabled on find_library, and having multiples names in the same directory is unlikely but might happen when installing a newer openssl over an old one I think. I will change to this : + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libssl ssleay32${_OPENSSL_MSVC_RT_MODE} ssleay32 ssl Also, I didn't see the intel case in FindOpenSSL.cmake, it also need changes as it actually only search for ssleay32 and libeay32. Alexis 2016-08-28 23:18 GMT+02:00 Craig Scott : > In that patch, the old names are listed before the new names, so if both > exist on the system, the older version would be found first. Would it be > preferable instead to swap the order to give preference to the newer version > in such a scenario? > > -- > Craig Scott > Melbourne, Australia > http://crascit.com From amubtdx at outlook.fr Mon Aug 29 14:55:43 2016 From: amubtdx at outlook.fr (Alexis Murzeau) Date: Mon, 29 Aug 2016 18:55:43 +0000 Subject: [cmake-developers] [PATCH] FindOpenSSL: Fix detection of OpenSSL 1.1 w/ MSVC In-Reply-To: References: Message-ID: NAMES_PER_DIR is enabled on find_library, and having multiples names in the same directory is unlikely but might happen when installing a newer openssl over an old one I think. I will change to this : + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libssl ssleay32${_OPENSSL_MSVC_RT_MODE} ssleay32 ssl Also, I didn't see the intel case in FindOpenSSL.cmake, it also need changes as it actually only search for ssleay32 and libeay32. Alexis 2016-08-28 23:18 GMT+02:00 Craig Scott : > In that patch, the old names are listed before the new names, so if both > exist on the system, the older version would be found first. Would it be > preferable instead to swap the order to give preference to the newer version > in such a scenario? > > -- > Craig Scott > Melbourne, Australia > http://crascit.com From amubtdx at outlook.fr Mon Aug 29 16:57:32 2016 From: amubtdx at outlook.fr (Alexis Murzeau) Date: Mon, 29 Aug 2016 20:57:32 +0000 Subject: [cmake-developers] [PATCH v2] FindOpenSSL: Fix detection of OpenSSL 1.1 Win32/64 Message-ID: Since OpenSSL 1.1.0, Windows binaries are libcrypto and libssl instead of the old names libeay32 and ssleay32. When using MSVC, FindOpenSSL was searching for the old lib names only so this add the new names to be able to find OpenSSL 1.1.0 libraries. For example, the files in lib directory of OpenSSL 1.1.0 Win64 : - libcrypto.lib - libssl.lib - VC/libcrypto64MD.lib - VC/libcrypto64MDd.lib - VC/libcrypto64MT.lib - VC/libcrypto64MTd.lib - VC/libssl64MD.lib - VC/libssl64MDd.lib - VC/libssl64MT.lib - VC/libssl64MTd.lib 32 bits OpenSSL has the same files with "32" instead of "64" for files in VC directory. MinGW still works and use lib/libcrypto.lib and lib/libssl.lib. This patch also add libssl and libcrypto for other windows compilers too (like Intel). --- Modules/FindOpenSSL.cmake | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/Modules/FindOpenSSL.cmake b/Modules/FindOpenSSL.cmake index 10b62ff..7ddd783 100644 --- a/Modules/FindOpenSSL.cmake +++ b/Modules/FindOpenSSL.cmake @@ -133,6 +133,13 @@ if(WIN32 AND NOT CYGWIN) set(_OPENSSL_MSVC_RT_MODE "MD") endif () + # Since OpenSSL 1.1, lib names are like libcrypto32MTd.lib and libssl32MTd.lib + if( "${CMAKE_SIZEOF_VOID_P}" STREQUAL "8" ) + set(_OPENSSL_MSVC_ARCH_SUFFIX "64") + else() + set(_OPENSSL_MSVC_ARCH_SUFFIX "32") + endif() + if(OPENSSL_USE_STATIC_LIBS) set(_OPENSSL_PATH_SUFFIXES "lib" @@ -149,6 +156,8 @@ if(WIN32 AND NOT CYGWIN) find_library(LIB_EAY_DEBUG NAMES + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libcryptod libeay32${_OPENSSL_MSVC_RT_MODE}d libeay32d NAMES_PER_DIR @@ -159,6 +168,8 @@ if(WIN32 AND NOT CYGWIN) find_library(LIB_EAY_RELEASE NAMES + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libcrypto libeay32${_OPENSSL_MSVC_RT_MODE} libeay32 NAMES_PER_DIR @@ -169,6 +180,8 @@ if(WIN32 AND NOT CYGWIN) find_library(SSL_EAY_DEBUG NAMES + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libssld ssleay32${_OPENSSL_MSVC_RT_MODE}d ssleay32d NAMES_PER_DIR @@ -179,6 +192,8 @@ if(WIN32 AND NOT CYGWIN) find_library(SSL_EAY_RELEASE NAMES + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libssl ssleay32${_OPENSSL_MSVC_RT_MODE} ssleay32 ssl @@ -236,6 +251,7 @@ if(WIN32 AND NOT CYGWIN) # Not sure what to pick for -say- intel, let's use the toplevel ones and hope someone report issues: find_library(LIB_EAY NAMES + libcrypto libeay32 NAMES_PER_DIR ${_OPENSSL_ROOT_HINTS_AND_PATHS} @@ -247,6 +263,7 @@ if(WIN32 AND NOT CYGWIN) find_library(SSL_EAY NAMES + libssl ssleay32 NAMES_PER_DIR ${_OPENSSL_ROOT_HINTS_AND_PATHS} -- 2.7.4.windows.1 From yves.frederix+cmake at gmail.com Tue Aug 30 03:58:42 2016 From: yves.frederix+cmake at gmail.com (Yves Frederix) Date: Tue, 30 Aug 2016 09:58:42 +0200 Subject: [cmake-developers] [PATCH] install: correctly deal with paths in generator expressions in DIRECTORY Message-ID: Hi, Some time ago, I contributed generator expressions to install(DIRECTORY). It turns out, however, that the patch was incomplete and that there are problems in case the provided directory starts with a generator expression and evaluates to a full path. In attachment you can find a fix for the problem. Regards, Yves -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-install-correctly-deal-with-paths-in-generator-expre.patch Type: application/octet-stream Size: 2207 bytes Desc: not available URL: From brad.king at kitware.com Tue Aug 30 10:53:35 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 30 Aug 2016 10:53:35 -0400 Subject: [cmake-developers] QtAutogen: Use Base32 instead of Base64 for file names In-Reply-To: References: Message-ID: On 08/28/2016 10:00 AM, Sebastian Holtermann wrote: > Base32 is a better choice IMO because it is single case and does not > generate disruptive characters like '_', '-' or '/'. > > Here are two patches that replace Base64 with Base32. Thanks. Please revise the Base32 infrastructure to simply be in CMake instead of KWSys. Thanks, -Brad From brad.king at kitware.com Tue Aug 30 10:53:45 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 30 Aug 2016 10:53:45 -0400 Subject: [cmake-developers] [patch] Expose GNUInstallDirs absolute path algorithm In-Reply-To: <552e0cb1-0ea8-802f-4710-ac1cd5b51e29@codelibre.net> References: <552e0cb1-0ea8-802f-4710-ac1cd5b51e29@codelibre.net> Message-ID: On 08/28/2016 01:51 PM, Roger Leigh wrote: > The macro name might need adjusting The idea looks fine to me. Typically we name module-provided APIs with the module name. How about "GNUInstallDirs_get_absolute_install_dir"? I'm also open to other suggestions. Thanks, -Brad From brad.king at kitware.com Tue Aug 30 10:53:54 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 30 Aug 2016 10:53:54 -0400 Subject: [cmake-developers] [PATCH] install: correctly deal with paths in generator expressions in DIRECTORY In-Reply-To: References: Message-ID: <9554381f-ce87-9990-05ac-9ff282d7ddc2@kitware.com> On 08/30/2016 03:58 AM, Yves Frederix wrote: > Some time ago, I contributed generator expressions to > install(DIRECTORY). It turns out, however, that the patch was > incomplete and that there are problems in case the provided directory > starts with a generator expression and evaluates to a full path. In > attachment you can find a fix for the problem. Thanks. > - if (!cmSystemTools::FileIsFullPath(dir.c_str())) { > + std::string::size_type gpos = cmGeneratorExpression::Find(dir); > + if (gpos != 0 && !cmSystemTools::FileIsFullPath(dir.c_str())) { Good. This check is used in several similar locations. > + if (!cmSystemTools::FileIsFullPath(i->c_str())) { > + *i = std::string(makefile.GetCurrentSourceDirectory()) + "/" + *i; Perhaps we should issue a diagnostic here. In principle the caller should always provide a full path. If they use a genex to do it then they are on their own. Thanks, -Brad From brad.king at kitware.com Tue Aug 30 10:54:04 2016 From: brad.king at kitware.com (Brad King) Date: Tue, 30 Aug 2016 10:54:04 -0400 Subject: [cmake-developers] [PATCH v2] FindOpenSSL: Fix detection of OpenSSL 1.1 Win32/64 In-Reply-To: References: Message-ID: On 08/29/2016 04:57 PM, Alexis Murzeau wrote: > Since OpenSSL 1.1.0, Windows binaries are libcrypto and libssl instead of > the old names libeay32 and ssleay32. Thanks, applied: FindOpenSSL: Fix detection of OpenSSL 1.1 Win32/64 https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=ed1758f8 -Brad From yves.frederix+cmake at gmail.com Wed Aug 31 03:41:07 2016 From: yves.frederix+cmake at gmail.com (Yves Frederix) Date: Wed, 31 Aug 2016 09:41:07 +0200 Subject: [cmake-developers] [PATCH] install: correctly deal with paths in generator expressions in DIRECTORY In-Reply-To: <9554381f-ce87-9990-05ac-9ff282d7ddc2@kitware.com> References: <9554381f-ce87-9990-05ac-9ff282d7ddc2@kitware.com> Message-ID: Hi Brad, >> + if (!cmSystemTools::FileIsFullPath(i->c_str())) { >> + *i = std::string(makefile.GetCurrentSourceDirectory()) + "/" + *i; > > Perhaps we should issue a diagnostic here. In principle the caller > should always provide a full path. If they use a genex to do it then > they are on their own. I don't really mind. The reason I did it like this was mainly that the docs for install(DIRECTORY) mention: "Directory names given as relative paths are interpreted with respect to the current source directory.". If we can have consistent and documented behavior without needing to issues warning or error messages, wouldn't it be better to do so? Additionally, the same 'fix' could also be applied to install(FILES), which in the docs currently explicitly states that "However, if any item begins in a generator expression it must evaluate to a full path". Please let me know what you decide and I'll prepare a new patch. Thanks, Yves From brad.king at kitware.com Wed Aug 31 09:38:30 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 31 Aug 2016 09:38:30 -0400 Subject: [cmake-developers] [PATCH] install: correctly deal with paths in generator expressions in DIRECTORY In-Reply-To: References: <9554381f-ce87-9990-05ac-9ff282d7ddc2@kitware.com> Message-ID: On 08/31/2016 03:41 AM, Yves Frederix wrote: > Please let me know what you decide Thanks. I decided to apply the patch as-is with minor tweaks: install: Fix evaluation of leading generator expressions in DIRECTORY https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3bd55dba -Brad From Stephen.Sorley at jhuapl.edu Wed Aug 31 10:50:50 2016 From: Stephen.Sorley at jhuapl.edu (Sorley, Stephen L.) Date: Wed, 31 Aug 2016 14:50:50 +0000 Subject: [cmake-developers] FindCUDA bug fixes Message-ID: I've attached two patches that fix bugs in the CUDA_COMPILE{,_PTX,_FATBIN,_CUBIN} macros from FindCUDA.cmake. First bug (fixed by patch #1) Commit 7ded655 added generator expressions in CUDA_WRAP_SRCS to scrape include directories and compile definitions off of the target. This works great when the target name passed to CUDA_WRAP_SRCS is an actual target (like in cuda_add_library and cuda_add_executable). However, the CUDA_COMPILE* macros also use CUDA_WRAP_SRCS, and they pass in a hardcoded name that doesn't represent a real target. This breaks the generator expressions, causing CMake to abort during generation. I fixed this by teaching CUDA_WRAP_SRCS to check for "PHONY" in its argument list. When CUDA_WRAP_SRCS sees "PHONY", it queries the appropriate directory properties (INCLUDE_DIRECTORIES and COMPILE_DEFINITIONS) instead of using the generator expressions. I then modified cuda_compile_base (which is used internally by all the CUDA_COMPILE* macros) to pass PHONY to CUDA_WRAP_SRCS. Second bug (fixed by patch #2) In a couple spots, CUDA_WRAP_SRCS assumes that the passed-in target name is unique - for example, the name of the directory containing the intermediate output is built from the target name. However, the CUDA_COMPILE* macros always pass the same hardcoded target name. So, if you call the same macro twice in a directory, some of the generated files from the second call will silently overwrite those from the first call. I fixed this by adding a counter (_cuda_internal_phony_counter) as a directory property. The counter gets incremented every time cuda_compile_base is called, and the value of the counter is appended to the hardcoded target name that gets passed to CUDA_WRAP_SRCS. This ensures that each call to the macro has its own unique target name. Thanks! Stephen Sorley -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-FindCUDA-Fix-for-broken-cuda_compile-commands.patch Type: application/octet-stream Size: 5044 bytes Desc: 0001-FindCUDA-Fix-for-broken-cuda_compile-commands.patch URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-FindCUDA-Allow-cuda_compile-macros-to-be-called-more.patch Type: application/octet-stream Size: 2002 bytes Desc: 0002-FindCUDA-Allow-cuda_compile-macros-to-be-called-more.patch URL: From timmerov+cmake at gmail.com Wed Aug 31 11:47:20 2016 From: timmerov+cmake at gmail.com (tim cotter) Date: Wed, 31 Aug 2016 08:47:20 -0700 Subject: [cmake-developers] Android.mk Message-ID: is anyone working on a generator for Android.mk files? i recently parted ways with a company that could have benefited mightily from the feature. i'm not planning to take up anything new for a while. so i might be able to contribute. maybe get a brain dump from someone who's thought about the problem. doesn't seem like it should be too hard. but maybe i'm being naive. am not too unfamiliar with the cmake source. been using it for a 10ish years. i didn't like the way files are grouped in codeblocks. so i hacked up a "better" solution. -timmer p.s. i found the previous Android.mk thread. but wasn't smart enough to figure out how to reply to it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad.king at kitware.com Wed Aug 31 11:51:42 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 31 Aug 2016 11:51:42 -0400 Subject: [cmake-developers] Android.mk In-Reply-To: References: Message-ID: On 08/31/2016 11:47 AM, tim cotter wrote: > is anyone working on a generator for Android.mk files? This has been investigated in the past and deemed hard/impossible since the Android.mk build system does not expose hooks we need. However, the main use case for this is to bring CMake-based projects into Android.mk builds. For that there is work here: https://gitlab.kitware.com/cmake/cmake/merge_requests/67 to create and install Android.mk files as part of the CMake build process. They reference the installed binaries as pre-built libs. -Brad From brad.king at kitware.com Wed Aug 31 11:53:42 2016 From: brad.king at kitware.com (Brad King) Date: Wed, 31 Aug 2016 11:53:42 -0400 Subject: [cmake-developers] FindCUDA bug fixes In-Reply-To: References: Message-ID: <812a4e75-193b-2d91-efdf-11b860a11b3e@kitware.com> On 08/31/2016 10:50 AM, Sorley, Stephen L. wrote: > I?ve attached two patches that fix bugs Thanks, applied: FindCUDA: Fix for broken cuda_compile* commands. https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=6442709b FindCUDA: Allow cuda_compile* macros to be called more than once per directory https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=900ee0b8 -Brad From timmerov+cmake at gmail.com Wed Aug 31 15:55:28 2016 From: timmerov+cmake at gmail.com (tim cotter) Date: Wed, 31 Aug 2016 12:55:28 -0700 Subject: [cmake-developers] Android.mk In-Reply-To: References: Message-ID: thanks! very cool. go brad go! build via toolchain didn't fly with my former employer. they required the code tree to be built by actual honest to god Android.mk files. don't ask me wy. companies do funny things sometimes. On Wed, Aug 31, 2016 at 8:51 AM, Brad King wrote: > On 08/31/2016 11:47 AM, tim cotter wrote: > > is anyone working on a generator for Android.mk files? > > This has been investigated in the past and deemed hard/impossible > since the Android.mk build system does not expose hooks we need. > > However, the main use case for this is to bring CMake-based projects > into Android.mk builds. For that there is work here: > > https://gitlab.kitware.com/cmake/cmake/merge_requests/67 > > to create and install Android.mk files as part of the CMake build > process. They reference the installed binaries as pre-built libs. > > -Brad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From craig.scott at crascit.com Wed Aug 31 20:50:12 2016 From: craig.scott at crascit.com (Craig Scott) Date: Thu, 1 Sep 2016 10:50:12 +1000 Subject: [cmake-developers] [CMake] Setup/tear down steps for CTest In-Reply-To: <20160825140810.GA13534@megas.kitware.com> References: <2790033.ITbBjc0OBD@caliban.sf-tec.de> <20160825140810.GA13534@megas.kitware.com> Message-ID: In my original thinking, I was of the view that if a setup/cleanup step needed to be executed for each test rather than for the overall test run as a whole, then perhaps the test itself should handle that rather than CMake. The existing RESOURCE_LOCK functionality could then be used to prevent multiple tests from running concurrently if they would interfere with each other. Existing test frameworks like GoogleTest and Boost Test already have good support for test fixtures which make doing this per-test setup/cleanup easy. The problem I want to solve is where a group of tests share a common (set of) setup/cleanup steps and CMake knows to run them when asked to run any test cases that require them. The specific problem motivating this work was running ctest --rerun-failed, where we need CMake to add in any setup/cleanup steps required by any of the tests that will be rerun. With that in mind, see further comments interspersed below. On Fri, Aug 26, 2016 at 12:08 AM, Ben Boeckel wrote: > On Tue, Aug 23, 2016 at 08:00:09 +0200, Rolf Eike Beer wrote: > > Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: > > > So how would you want the feature to work? I'd suggest an initial set > of > > > requirements something like the following: > > > > > > - Need to support the ability to define multiple setup and/or tear > down > > > tasks. > > > - It should be possible to specify dependencies between setup tasks > and > > > between tear down tasks. > > > - Individual tests need to be able to indicate which setup and/or > tear > > > down tasks they require, similar to the way DEPENDS is used to > specify > > > dependencies between test cases. > > > - When using ctest --rerun-failed, ctest should automatically > invoke any > > > setup or tear down tasks required by the test cases that will be > re-run. > > > - Setup or tear down tasks which reference executable targets should > > > substitute the actual built executable just like how > add_custom_command() > > > does. > > > > -need a way to mark if 2 tests with the same setup/teardown can share > those or > > if they need to run for every of them > > Proposal: > > add_test(NAME setup-foo ...) > set_tests_properties(setup-foo PROPERTIES > SETUP_GROUP foo > SETUP_STEP SETUP_PER_TEST) # Also SETUP_ONCE. > add_test(NAME use-foo ...) > set_tests_properties(use-foo PROPERTIES > SETUP_GROUP foo) # implicit depends on all SETUP_GROUP foo / > SETUP_STEP SETUP_* tests. > add_test(NAME use-foo2 ...) > set_tests_properties(use-foo2 PROPERTIES > SETUP_GROUP foo) > add_test(NAME teardown-foo2 ...) > set_tests_properties(teardown-foo2 PROPERTIES > SETUP_GROUP foo > SETUP_STEP TEARDOWN) # implicit depends on all non-TEARDOWN steps > > Multiple setup/teardown steps could be done with DEPENDS between them. > I like the idea of tests being associated with a group and the group itself is where the setup/cleanup steps are attached/associated. That said, it would seem that RESOURCE_LOCK already more or less satisfies this concept. I'm wondering if we can't just somehow attach setup/cleanup steps to the named resource instead. That would be a more seamless evolution of the existing functionality and have little impact on any existing code. Basically all we'd need to do is add the ability to associate the setup/cleanup steps with a RESOURCE_LOCK label. It's still not clear to me whether the setup/cleanup tasks should be considered test cases themselves, but I can see benefits with taking that path. It would mean all we'd need is to be able to mark a test case as "this is a setup/cleanup step for RESOURCE_LOCK label XXX", maybe something like this: set_tests_properties(setup-foo PROPERTIES RESOURCE_SETUP foo) set_tests_properties(teardown-foo PROPERTIES RESOURCE_CLEANUP foo) If multiple setup/cleanup steps are defined for a particular resource, then dependencies between those test cases would determine their order and where there are no dependencies, the order would be undefined as is already the case for test cases. For the initial implementation at least, I think something like the SETUP_PER_TEST concept is more complexity than I'd want to tackle. Maybe it could be supported later, but in the first instance I think once per group/resource is already a significant win and worth focusing on at the start (see my motivation at the top of this email). > > > -the default for each test is "no s/t", which means it can't be run with > any > > of the above in parallel (especially for compatibillity)[1] > > -need a way to tell if a test doesn't care about those > > Making RESOURCE_LOCK a rwlock rather than a mutex might make sense here. > SETUP_STEP bits have a RESOURCE_LOCK_WRITE group_${group}, otherwise it > is RESOURCE_LOCK_READ group_${group}. > Not sure I follow what problem this solves and without a strong motivation, I'd be reluctant to add this sort of complexity to the existing RESOURCE_LOCK functionality. It's currently quite clean and easy to understand. If a test uses some resource, it specifies it in RESOURCE_LOCK. The proposal above to add setup/cleanup logic to a resource doesn't require differentiating readers and writers (but I'm happy to consider examples which do demonstrate the need). > > > 1) think of a database connector test: the test that will check what > happens > > if no DB is present will fail if the setup step "start DB" was run, but > not > > the teardown > > RESOURCE_LOCK on that group_${group} can fix that I think. > And this is indeed precisely the motivating situation that got me into this thread. We currently use the RESOURCE_LOCK to prevent concurrent access to a DB instance, with starting up a clean instance at the beginning and shutting it down again at the end of all tests being what I want to move into the proposed setup/cleanup tasks. The current functionality requires us to use both RESOURCE_LOCK and DEPENDS to specify the same thing and it doesn't cover the ctest --rerun-failed scenario. With the proposal above to use RESOURCE_SETUP and RESOURCE_CLEANUP test properties, this could create an implicit dependency on those setup/cleanup test cases just by using RESOURCE_LOCK on the test cases which use that resource (i.e. no need for the separate DEPENDS to be specified as it does now). -- Craig Scott Melbourne, Australia http://crascit.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From craig.scott at crascit.com Wed Aug 31 20:59:47 2016 From: craig.scott at crascit.com (Craig Scott) Date: Thu, 1 Sep 2016 10:59:47 +1000 Subject: [cmake-developers] [CMake] Setup/tear down steps for CTest In-Reply-To: <2790033.ITbBjc0OBD@caliban.sf-tec.de> References: <2790033.ITbBjc0OBD@caliban.sf-tec.de> Message-ID: On Tue, Aug 23, 2016 at 4:00 PM, Rolf Eike Beer wrote: > Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: > > Cheeky way to get me more involved in contributing, but okay, I'll bite. > ;) > > Switching discussion to the dev list. > > > > So how would you want the feature to work? I'd suggest an initial set of > > requirements something like the following: > > > > - Need to support the ability to define multiple setup and/or tear > down > > tasks. > > - It should be possible to specify dependencies between setup tasks > and > > between tear down tasks. > > - Individual tests need to be able to indicate which setup and/or tear > > down tasks they require, similar to the way DEPENDS is used to specify > > dependencies between test cases. > > - When using ctest --rerun-failed, ctest should automatically invoke > any > > setup or tear down tasks required by the test cases that will be > re-run. > > - Setup or tear down tasks which reference executable targets should > > substitute the actual built executable just like how > add_custom_command() > > does. > > -need a way to mark if 2 tests with the same setup/teardown can share > those or > if they need to run for every of them > -the default for each test is "no s/t", which means it can't be run with > any > of the above in parallel (especially for compatibillity)[1] > -need a way to tell if a test doesn't care about those > > 1) think of a database connector test: the test that will check what > happens > if no DB is present will fail if the setup step "start DB" was run, but not > the teardown > So maybe that requires being able to specify that tests for resource XXX and resource YYY cannot be executed concurrently. Maybe that's a separate change that could be made independent of this proposed improvement, since it would apply even for existing CMake functionality. I see the value, I'm just trying to sort out what is really needed from what is nice-to-have but could be done as a subsequent improvement later. > > > Some open questions: > > > > - Should setup and tear down tasks be defined in pairs, or should they > > completely independent (this would still require the ability to > specify a > > dependency of a tear down task on a setup task)? > > The test could be "shutdown daemon" or "delete files" so I would keep them > separated. They may be created by the same command, so they could be > batched > anyway. > Agreed, it seems clear now that keeping them separate is preferable. > > > - Should the setup and tear down tasks be defined by a new CTest/CMake > > command or extend an existing mechanism (e.g. add_custom_command())? > > Don't clutter existing commands more than needed. If it's something new, > then > create a new command (they could still share C++ code). If it's basically > the > same as anything existing at the end then use that. > See my other email reply just now. I think re-using the existing commands and concepts and adding the RESOURCE_SETUP and RESOURCE_CLEANUP test properties might be the most seamless from an end user perspective. I might change my mind once I dig into the CMake source code though. ;) > > > - If no test case has a dependency on a setup or tear down task, > should > > that task be skipped? Perhaps tasks need to have a flag which > indicates > > whether they always run or only if a test case depends on it. > > Keep backward compatibility. I.e. if I now add a new test with s/t, then > every > other test should still run (and succeed) as before. > Definitely. Existing projects should receive zero impact from any changes made. New functionality should be opt-in. > > > - What terminology to use? Things like GoogleTest use terms like test > > *fixtures* for this sort of thing. The terms setup and tear down are a > > bit imprecise and cumbersome, so we would probably need something > better > > than those. > > - Would it make sense for the ctest command line to support disabling > > setup and/or tear down steps? I can see some potential scenarios where > > this may be desirable, but maybe this is getting too ambitious for a > > starting set of requirements. > > IMHO that doesn't make sense. One could think about an option to disable > the > s/t merging, i.e. that they are really called alone for every test. > To reduce complexity, I'm gravitating that way too. If you define a setup/cleanup task, then why allow disabling it? If developers really want that, they could wrap defining the setup/cleanup definitions inside an if() block controlled by a CMake option or something similar. > > > - What should happen if a setup or tear down task fails? How would > > failure be detected? How would such failures impact things like a > CDash > > test report, etc.? > > Then the test fails, just the same as it now does when it can't find the > executable. > Seems sensible. -- Craig Scott Melbourne, Australia http://crascit.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From craig.scott at crascit.com Wed Aug 31 21:17:49 2016 From: craig.scott at crascit.com (Craig Scott) Date: Thu, 1 Sep 2016 11:17:49 +1000 Subject: [cmake-developers] [CMake] Setup/tear down steps for CTest In-Reply-To: References: <2790033.ITbBjc0OBD@caliban.sf-tec.de> <20160825140810.GA13534@megas.kitware.com> Message-ID: Actually, we can't really re-use the RESOURCE_LOCK for the proposed RESOURCE_SETUP and RESOURCE_CLEANUP functionality since that would force all the tests using that resource to be serialised. So yes, a separate GROUP or something similar would seem to be needed. Let me amend my earlier proposal (which is an evolution of Ben's) to something like this: add_test(NAME setup-foo ...) set_tests_properties(setup-foo PROPERTIES GROUP_SETUP foo) add_test(NAME cleanup-foo ...) set_tests_properties(cleanup-foo PROPERTIES GROUP_CLEANUP foo) add_test(NAME use-foo ...) set_tests_properties(use-foo PROPERTIES GROUP foo) The logic would be as follows: - Any test cases with a GROUP_SETUP property for a group will be run before any test cases with GROUP or GROUP_CLEANUP for that same group. The order of these setup test cases can be controlled with the existing DEPENDS test property. - If any of the group's setup test cases fail, all other test cases for that group will be skipped. All cleanup test cases for the group probably should still be run though (it could be hard to try to work out which cleanup tests should run, so maybe conservatively just run all of them). - If all setup test cases passed, then run all test cases for that group. Regardless of the success or failure of these test cases, once they are all completed, run all the cleanup test cases associated with the group. - Ordering of cleanup test cases can again be controlled with the existing DEPENDS test property. What the above buys us is that CTest then knows definitively that if it is asked to run a test case from a particular group, it must also run the setup and cleanup test cases associated with that group, regardless of whether those setup/cleanup test cases are in the set of test cases CTest was originally asked to run. At the moment, CTest could theoretically do that for the setup steps based on DEPENDS functionality, but not the cleanup. The above proposal is very clear about the nature of the dependency and gives the symmetry of both setup and cleanup behaviour. I'm not tied to the terminology of "GROUP" for tying a set of test cases to their setup/cleanup tasks, so I'm happy to consider alternatives. I'm also wondering whether simply GROUP for a test property is too generic for the test cases that require the setup/cleanup (as opposed to the test cases that ARE the setup/cleanup). On Thu, Sep 1, 2016 at 10:50 AM, Craig Scott wrote: > In my original thinking, I was of the view that if a setup/cleanup step > needed to be executed for each test rather than for the overall test run as > a whole, then perhaps the test itself should handle that rather than CMake. > The existing RESOURCE_LOCK functionality could then be used to prevent > multiple tests from running concurrently if they would interfere with each > other. Existing test frameworks like GoogleTest and Boost Test already have > good support for test fixtures which make doing this per-test setup/cleanup > easy. The problem I want to solve is where a group of tests share a common > (set of) setup/cleanup steps and CMake knows to run them when asked to run > any test cases that require them. The specific problem motivating this work > was running ctest --rerun-failed, where we need CMake to add in any > setup/cleanup steps required by any of the tests that will be rerun. With > that in mind, see further comments interspersed below. > > > On Fri, Aug 26, 2016 at 12:08 AM, Ben Boeckel > wrote: > >> On Tue, Aug 23, 2016 at 08:00:09 +0200, Rolf Eike Beer wrote: >> > Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: >> > > So how would you want the feature to work? I'd suggest an initial set >> of >> > > requirements something like the following: >> > > >> > > - Need to support the ability to define multiple setup and/or tear >> down >> > > tasks. >> > > - It should be possible to specify dependencies between setup >> tasks and >> > > between tear down tasks. >> > > - Individual tests need to be able to indicate which setup and/or >> tear >> > > down tasks they require, similar to the way DEPENDS is used to >> specify >> > > dependencies between test cases. >> > > - When using ctest --rerun-failed, ctest should automatically >> invoke any >> > > setup or tear down tasks required by the test cases that will be >> re-run. >> > > - Setup or tear down tasks which reference executable targets >> should >> > > substitute the actual built executable just like how >> add_custom_command() >> > > does. >> > >> > -need a way to mark if 2 tests with the same setup/teardown can share >> those or >> > if they need to run for every of them >> >> Proposal: >> >> add_test(NAME setup-foo ...) >> set_tests_properties(setup-foo PROPERTIES >> SETUP_GROUP foo >> SETUP_STEP SETUP_PER_TEST) # Also SETUP_ONCE. >> add_test(NAME use-foo ...) >> set_tests_properties(use-foo PROPERTIES >> SETUP_GROUP foo) # implicit depends on all SETUP_GROUP foo / >> SETUP_STEP SETUP_* tests. >> add_test(NAME use-foo2 ...) >> set_tests_properties(use-foo2 PROPERTIES >> SETUP_GROUP foo) >> add_test(NAME teardown-foo2 ...) >> set_tests_properties(teardown-foo2 PROPERTIES >> SETUP_GROUP foo >> SETUP_STEP TEARDOWN) # implicit depends on all non-TEARDOWN steps >> >> Multiple setup/teardown steps could be done with DEPENDS between them. >> > > I like the idea of tests being associated with a group and the group > itself is where the setup/cleanup steps are attached/associated. That said, > it would seem that RESOURCE_LOCK already more or less satisfies this > concept. I'm wondering if we can't just somehow attach setup/cleanup steps > to the named resource instead. That would be a more seamless evolution of > the existing functionality and have little impact on any existing code. > Basically all we'd need to do is add the ability to associate the > setup/cleanup steps with a RESOURCE_LOCK label. > > It's still not clear to me whether the setup/cleanup tasks should be > considered test cases themselves, but I can see benefits with taking that > path. It would mean all we'd need is to be able to mark a test case as > "this is a setup/cleanup step for RESOURCE_LOCK label XXX", maybe something > like this: > > set_tests_properties(setup-foo PROPERTIES RESOURCE_SETUP foo) > set_tests_properties(teardown-foo PROPERTIES RESOURCE_CLEANUP foo) > > If multiple setup/cleanup steps are defined for a particular resource, > then dependencies between those test cases would determine their order and > where there are no dependencies, the order would be undefined as is already > the case for test cases. > > For the initial implementation at least, I think something like the > SETUP_PER_TEST concept is more complexity than I'd want to tackle. Maybe it > could be supported later, but in the first instance I think once per > group/resource is already a significant win and worth focusing on at the > start (see my motivation at the top of this email). > > > >> >> > -the default for each test is "no s/t", which means it can't be run >> with any >> > of the above in parallel (especially for compatibillity)[1] >> > -need a way to tell if a test doesn't care about those >> >> Making RESOURCE_LOCK a rwlock rather than a mutex might make sense here. >> SETUP_STEP bits have a RESOURCE_LOCK_WRITE group_${group}, otherwise it >> is RESOURCE_LOCK_READ group_${group}. >> > > Not sure I follow what problem this solves and without a strong > motivation, I'd be reluctant to add this sort of complexity to the existing > RESOURCE_LOCK functionality. It's currently quite clean and easy to > understand. If a test uses some resource, it specifies it in RESOURCE_LOCK. > The proposal above to add setup/cleanup logic to a resource doesn't require > differentiating readers and writers (but I'm happy to consider examples > which do demonstrate the need). > > >> >> > 1) think of a database connector test: the test that will check what >> happens >> > if no DB is present will fail if the setup step "start DB" was run, but >> not >> > the teardown >> >> RESOURCE_LOCK on that group_${group} can fix that I think. >> > > And this is indeed precisely the motivating situation that got me into > this thread. We currently use the RESOURCE_LOCK to prevent concurrent > access to a DB instance, with starting up a clean instance at the beginning > and shutting it down again at the end of all tests being what I want to > move into the proposed setup/cleanup tasks. The current functionality > requires us to use both RESOURCE_LOCK and DEPENDS to specify the same thing > and it doesn't cover the ctest --rerun-failed scenario. With the proposal > above to use RESOURCE_SETUP and RESOURCE_CLEANUP test properties, this > could create an implicit dependency on those setup/cleanup test cases just > by using RESOURCE_LOCK on the test cases which use that resource (i.e. no > need for the separate DEPENDS to be specified as it does now). > > > -- > Craig Scott > Melbourne, Australia > http://crascit.com > -- Craig Scott Melbourne, Australia http://crascit.com -------------- next part -------------- An HTML attachment was scrubbed... URL: