Hi,<div>I am no CMake expert, but I would test on an identical file system types...</div><div>I am pretty sure that ntfs and ext3fs don't have the same read-write access latency and this might explain your time difference.</div>
<div>You could additionally launch a strace on the two settings and see if there are major differences that could explain the performance hit...</div><div>Good luck.</div><div><br><div class="gmail_quote">On Sat, Sep 1, 2012 at 5:22 PM, Carlo Wood <span dir="ltr"><<a href="mailto:carlo@alinoe.com" target="_blank">carlo@alinoe.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello all.<br>
<br>
Apart from that I think that needing 7 seconds is too long<br>
for generating dependencies for source files that can be<br>
read in 1 second (and 1 second to write the result to disk),<br>
I ran into something that I cannot explain :/.<br>
<br>
I recently bought a new PC: 8 cores, 32 GB ram -- each core<br>
is faster than the 4 cores machine with 4 GB that I had;<br>
They run the same OS, same linux kernel, same software<br>
versions.<br>
<br>
SUMMARY OF RESULTS<br>
------------------<br>
<br>
The old 4 core machine (Intel QX6700 @2.6 GHz), generates the<br>
dependencies for this project in<br>
<br>
(OLD machine)<br>
<br>
real 0m7.601s<br>
user 0m2.488s<br>
sys 0m4.164s<br>
<br>
Which is annoying slow, but<br>
the new 8 core machine (AMD FX-8150 @3.6 GHz), generates<br>
the same dependencies for this project in<br>
<br>
(NEW machine)<br>
<br>
real 0m32.653s<br>
user 0m2.512s<br>
sys 0m10.385s<br>
<br>
which is unacceptably slow :/<br>
<br>
DETAILS<br>
-------<br>
<br>
$ cat /etc/debian_version<br>
OLD: wheezy/sid<br>
NEW: wheezy/sid<br>
<br>
$ uname -a<br>
OLD: Linux hikaru 3.2.0-3-amd64 #1 SMP Mon Jul 23 02:45:17 UTC 2012 x86_64 GNU/Linux<br>
NEW: Linux malatos 3.2.0-3-amd64 #1 SMP Mon Jul 23 02:45:17 UTC 2012 x86_64 GNU/Linux<br>
<br>
$ ldd /usr/bin/cmake<br>
OLD:<br>
linux-vdso.so.1 => (0x00007fff48372000)<br>
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fabcd035000)<br>
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fabcce0b000)<br>
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fabccbf3000)<br>
libarchive.so.12 => /usr/lib/x86_64-linux-gnu/libarchive.so.12 (0x00007fabcc957000)<br>
libcurl-gnutls.so.4 => /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007fabcc6f3000)<br>
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fabcc3eb000)<br>
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fabcc169000)<br>
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fabcbf53000)<br>
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fabcbbcb000)<br>
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fabcb9af000)<br>
/lib64/ld-linux-x86-64.so.2 (0x00007fabcd276000)<br>
libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007fabcb7a6000)<br>
libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007fabcb5a0000)<br>
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fabcb37d000)<br>
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fabcb16d000)<br>
libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007fabcae0d000)<br>
libnettle.so.4 => /usr/lib/x86_64-linux-gnu/libnettle.so.4 (0x00007fabcabe6000)<br>
libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007fabca9b2000)<br>
libssh2.so.1 => /usr/lib/x86_64-linux-gnu/libssh2.so.1 (0x00007fabca788000)<br>
liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007fabca579000)<br>
libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007fabca328000)<br>
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fabca11f000)<br>
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fabc9ee0000)<br>
libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007fabc9c20000)<br>
libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007fabc99a1000)<br>
librtmp.so.0 => /usr/lib/x86_64-linux-gnu/librtmp.so.0 (0x00007fabc9787000)<br>
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fabc9570000)<br>
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fabc9355000)<br>
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fabc9081000)<br>
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fabc8e57000)<br>
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fabc8c53000)<br>
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fabc8a4a000)<br>
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fabc8845000)<br>
libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x00007fabc8634000)<br>
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007fabc8422000)<br>
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007fabc821e000)<br>
NEW:<br>
linux-vdso.so.1 => (0x00007fffe5800000)<br>
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbd2bec0000)<br>
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fbd2bc90000)<br>
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fbd2ba78000)<br>
libarchive.so.12 => /usr/lib/x86_64-linux-gnu/libarchive.so.12 (0x00007fbd2b7d8000)<br>
libcurl-gnutls.so.4 => /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007fbd2b570000)<br>
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fbd2b268000)<br>
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbd2afe0000)<br>
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fbd2adc8000)<br>
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbd2aa40000)<br>
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbd2a820000)<br>
/lib64/ld-linux-x86-64.so.2 (0x00007fbd2c0e8000)<br>
libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007fbd2a610000)<br>
libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007fbd2a408000)<br>
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fbd2a1e0000)<br>
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fbd29fd0000)<br>
libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007fbd29c70000)<br>
libnettle.so.4 => /usr/lib/x86_64-linux-gnu/libnettle.so.4 (0x00007fbd29a48000)<br>
libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007fbd29810000)<br>
libssh2.so.1 => /usr/lib/x86_64-linux-gnu/libssh2.so.1 (0x00007fbd295e0000)<br>
liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007fbd293d0000)<br>
libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007fbd29178000)<br>
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbd28f70000)<br>
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fbd28d30000)<br>
libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x00007fbd28a70000)<br>
libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007fbd287f0000)<br>
librtmp.so.0 => /usr/lib/x86_64-linux-gnu/librtmp.so.0 (0x00007fbd285d0000)<br>
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fbd283b8000)<br>
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fbd28198000)<br>
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fbd27ec0000)<br>
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fbd27c90000)<br>
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fbd27a88000)<br>
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fbd27878000)<br>
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fbd27670000)<br>
libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x00007fbd27458000)<br>
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007fbd27240000)<br>
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007fbd27038000)<br>
<br>
$ cmake --version<br>
OLD: cmake version 2.8.9-rc1<br>
NEW: cmake version 2.8.9-rc1<br>
<br>
The executed commands in this case are:<br>
1) Remove the whole build directory.<br>
2) Reconfigure the project.<br>
3) Run the command that was timed above:<br>
<br>
OLD:<br>
<br>
time (cd /SSD/singularity/viewer-linux-x86_64-release; /usr/bin/cmake -E cmake_depends<br>
"Unix Makefiles" /usr/src/secondlife/viewers/singularity/SingularityViewer/linden/indra<br>
/usr/src/secondlife/viewers/singularity/SingularityViewer/linden/indra/newview<br>
/SSD/singularity/viewer-linux-x86_64-release<br>
/SSD/singularity/viewer-linux-x86_64-release/newview<br>
/SSD/singularity/viewer-linux-x86_64-release/newview/CMakeFiles/secondlife-bin.dir/DependInfo.cmake<br>
--color=)<br>
<br>
NEW:<br>
<br>
time (cd /SSD2/singularity/viewer-linux-x86_64-release; /usr/bin/cmake -E cmake_depends<br>
"Unix Makefiles" /opt-ntfs/secondlife/viewers/singularity/SingularityViewer/linden/indra<br>
/opt-ntfs/secondlife/viewers/singularity/SingularityViewer/linden/indra/newview<br>
/SSD2/singularity/viewer-linux-x86_64-release<br>
/SSD2/singularity/viewer-linux-x86_64-release/newview<br>
/SSD2/singularity/viewer-linux-x86_64-release/newview/CMakeFiles/secondlife-bin.dir/DependInfo.cmake<br>
--color=)<br>
<br>
(all on one line)<br>
<br>
<br>
The /SSD on the OLD machine is a mount to<br>
Filesystem Size Used Avail Use% Mounted on<br>
/dev/sdf1 127G 22G 105G 18% /SSD<br>
where<br>
$ ls -l /dev/disk/by-id | grep sdf1 | grep scsi<br>
lrwxrwxrwx 1 root root 10 Aug 30 20:40 scsi-SATA_Corsair_Perform1101810001000341012E-part1 -> ../../sdf1<br>
which is a 128 GB SSD.<br>
$ sudo hdparm -t -T /dev/sdf1<br>
gives 241.82 MB/sec read speed.<br>
<br>
The /SSD2 on the NEW machine is a mount to<br>
Filesystem Size Used Avail Use% Mounted on<br>
/dev/sdb3 60G 5.3G 51G 10% /SSD2<br>
where<br>
$ ls -l /dev/disk/by-id | grep sdb3 | grep scsi<br>
lrwxrwxrwx 1 root root 10 Aug 30 22:06 scsi-SATA_OCZ-VERTEX4_OCZ-A58A63H04CI286B9-part3 -> ../../sdb3<br>
which is a better 128 GB SSD.<br>
$ sudo hdparm -t -T /dev/sdb3<br>
gives 433.44 MB/sec read speed.<br>
<br>
The write speed a (a lot) faster too.<br>
<br>
I ran 'valgrind --tool=callgrind' on both commands, and they look the same.<br>
<br>
The output file size is:<br>
<br>
NEW:<br>
-rw-r--r-- 1 carlo carlo 22078131 Sep 1 05:08 /SSD2/singularity/viewer-linux-x86_64-release/newview/CMakeFiles/secondlife-bin.dir/depend.make<br>
OLD:<br>
-rw-r--r-- 1 carlo carlo 21944265 Sep 1 07:07 /SSD/singularity/viewer-linux-x86_64-release/newview/CMakeFiles/secondlife-bin.dir/depend.make<br>
<br>
where the difference is caused by the difference in paths (see above).<br>
<br>
<br>
So, if there is ANY difference at all, it has to be this you'd think:<br>
<br>
On the OLD machine /usr/src is a mount to:<br>
Filesystem Size Used Avail Use% Mounted on<br>
/dev/md7 60G 52G 5.0G 92% /usr/src<br>
where md7 is a RAID 5 of three HDD's: sdb, sdc and sdd, which<br>
are 10,000rpm WD Raptors's of 74.4 GB<br>
and md7 has an ext3 fs.<br>
hdparm -t -T /dev/md7 reports 152.17 MB/sec<br>
<br>
On the NEW machine /opt-ntfs is a mount to:<br>
Filesystem Size Used Avail Use% Mounted on<br>
/dev/sdd3 301G 2.3G 298G 1% /opt-ntfs<br>
lrwxrwxrwx 1 root root 10 Aug 30 22:03<br>
scsi-SATA_WDC_WD5000AAKX-_WD-WCAYUJX25063-part3 -> ../../sdd3<br>
which is a WD Caviar Blue 500 GB 7200rpm 16MB SATA3<br>
and has a NTFS.<br>
hdparm -t -T /dev/sdd3 reports 117.77 MB/sec<br>
<br>
Reading all source files (using cat, and after running hdparm -f on all drives)<br>
and writing them to /dev/null gives:<br>
<br>
OLD: 0.133 seconds<br>
NEW: 0.333 seconds<br>
<br>
Also note that pre-caching the source files<br>
by reading them all prior to running the cmake command makes<br>
hardly a difference in the timings of the latter.<br>
<br>
In other words, this can't have anything to do with reading<br>
the source code from disk.<br>
<br>
Any ideas what else I can test?<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Carlo Wood <<a href="mailto:carlo@alinoe.com">carlo@alinoe.com</a>><br>
--<br>
<br>
Powered by <a href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a href="http://www.kitware.com/opensource/opensource.html" target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the CMake FAQ at: <a href="http://www.cmake.org/Wiki/CMake_FAQ" target="_blank">http://www.cmake.org/Wiki/CMake_FAQ</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a href="http://www.cmake.org/mailman/listinfo/cmake" target="_blank">http://www.cmake.org/mailman/listinfo/cmake</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br> Amine<br>
</div>