View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008826CMakeCMakepublic2009-03-31 14:422009-09-22 13:05
ReporterWojciech Migda 
Assigned ToBrad King 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product VersionCMake-2-6 
Target VersionFixed in Version 
Summary0008826: CMake performance improvement on Cygwin
Description1. Cygwin is known for its poor performance with regard to some i/o operations, due to necessity to maintain *nix-like access rights management on ntfs filesystem.
2. there are know approaches to tackle this problem, such as recent improvements in git (http://kerneltrap.org/index.php?q=mailarchive/git/2008/9/27/3427614 [^])
3. CMake profiling during cmake_depends option execution shows that indeed calls to POSIX access function is the main bottleneck.
4. Simple benchmarking (approx. 700 files in a single directory, 80000 random executions of access) shows that it takes around 36 secs on cygwin/ntfs system regardless of the full/relative path being passed as access' argument.
5. the main source of calls to POSIX access in CMake is a function checking for file existence: SystemTools::FileExists
6. since on Cygwin file can be read even if a user has no reading rights replacing access with GetFileAttributesExA can be made to check for file existence. This requires conversion of cygwin paths to native Win32 form using a cygwin_conv_to_win32_path utility provided with the cygwin dll
7. this change alone brings drastic improvements in CMake performance during cmake_depends execution, however, the next bottleneck becomes the cygwin_conv_to_win32_path function. Fortunately we can cache path translations what moves this problem aside.
8. the attached patches for files cmake-2.6.3/Source/kwsys/SystemTools.cxx and cmake-2.6.3/Source/kwsys/SystemTools.hxx.in attempt to introduce basic changes to the way CMake checks for file existence. For one of the targets we have in our build system (ca. 130 .c files, 78,000 executions of access to check existnce of some of almost 700 headers) this cahnge has reduced cmake_depends timing from 36 secs to ca. 6 secs !!
Tagssolution
Attached Fileszip file icon patches.zip [^] (2,075 bytes) 2009-03-31 14:42

 Relationships

  Notes
(0017704)
Brad King (manager)
2009-09-22 13:05

 I've committed changes based on your patch. I kept the cygwin->windows path mapping and cache code. I re-wrote SystemTools::FileExists almost completely. It is now optimized on native Windows by using GetFileAttributesEx directly (instead of MSVC's access emulation API), and on cygwin by using your approach.

Optimize KWSys SystemTools::FileExists on Windows
/cvsroot/CMake/CMake/Source/kwsys/SystemTools.cxx,v <-- Source/kwsys/SystemTools.cxx
new revision: 1.249; previous revision: 1.248
/cvsroot/CMake/CMake/Source/kwsys/SystemTools.hxx.in,v <-- Source/kwsys/SystemTools.hxx.in
new revision: 1.79; previous revision: 1.78

 Issue History
Date Modified Username Field Change
2009-03-31 14:42 Wojciech Migda New Issue
2009-03-31 14:42 Wojciech Migda File Added: patches.zip
2009-03-31 16:29 Wojciech Migda Tag Attached: solution
2009-09-11 17:40 Bill Hoffman Status new => assigned
2009-09-11 17:40 Bill Hoffman Assigned To => Bill Hoffman
2009-09-11 17:41 Bill Hoffman Assigned To Bill Hoffman => Brad King
2009-09-22 13:05 Brad King Note Added: 0017704
2009-09-22 13:05 Brad King Status assigned => closed
2009-09-22 13:05 Brad King Resolution open => fixed

Copyright © 2000 - 2018 MantisBT Team