[cmake-developers] [CMake 0012618]: Slow execution of tests

[Mantis Bug Tracker]

The following issue has been SUBMITTED. 
====================================================================== 
http://public.kitware.com/Bug/view.php?id=12618 
====================================================================== 
Reported By:                Sébastien Valat
Assigned To:                
====================================================================== 
Project:                    CMake
Issue ID:                   12618
Category:                   CTest
Reproducibility:            always
Severity:                   minor
Priority:                   normal
Status:                     new
====================================================================== 
Date Submitted:             2011-12-10 16:58 EST
Last Modified:              2011-12-10 16:58 EST
====================================================================== 
Summary:                    Slow execution of tests
Description: 
Tests execution is very slow if we didn't use CDash.

time make test
Running tests...
Test project ....../project/build
      Start  1: svutDefaultMain
 1/36 Test  http://public.kitware.com/Bug/view.php?id=1: svutDefaultMain
..................   Passed    0.01 sec
      Start  2: svutDefaultMainXml
 2/36 Test  http://public.kitware.com/Bug/view.php?id=2: svutDefaultMainXml
...............   Passed    0.00 sec
      Start  3: svutDefaultMainStd
 3/36 Test  http://public.kitware.com/Bug/view.php?id=3: svutDefaultMainStd
...............   Passed    0.00 sec
      Start  4: svutDefaultMainStdColor
 4/36 Test  http://public.kitware.com/Bug/view.php?id=4: svutDefaultMainStdColor
..........   Passed    0.00 sec
      Start  5: svutStandeloneMain
 5/36 Test  http://public.kitware.com/Bug/view.php?id=5: svutStandeloneMain
...............   Passed    0.01 sec
      Start  6: svutStandeloneMainXml
 6/36 Test  http://public.kitware.com/Bug/view.php?id=6: svutStandeloneMainXml
............   Passed    0.00 sec
...... (the longer one is 0.03 sec)....
.......

real    0m10.473s
user    0m0.060s
sys     0m0.043s

So take 10 seconds to execute 36 tests of ~0.01 seconds. The longer was to wait
passing the first test.

While tracing in GDB when waiting on first test :

http://public.kitware.com/Bug/view.php?id=0  0x00007fb64469ffff in pthread_join
() from /lib/libpthread.so.0
http://public.kitware.com/Bug/view.php?id=1  0x00007fb6451a6afe in
Curl_thread_join () from /usr/lib/libcurl.so.4
http://public.kitware.com/Bug/view.php?id=2  0x00007fb6451a7ead in
Curl_resolver_wait_resolv () from /usr/lib/libcurl.so.4
http://public.kitware.com/Bug/view.php?id=3  0x00007fb64518e53c in ?? () from
/usr/lib/libcurl.so.4
http://public.kitware.com/Bug/view.php?id=4  0x000000000048d59d in
cmCTest::HTTPRequest(std::string, cmCTest::HTTPMethod, std::string&,
std::string, std::string, int) ()
http://public.kitware.com/Bug/view.php?id=5  0x0000000000493ed0 in
cmCTest::GetCDashVersion() ()
http://public.kitware.com/Bug/view.php?id=6  0x000000000049408b in
cmCTest::ShouldCompressTestOutput() ()
http://public.kitware.com/Bug/view.php?id=7  0x000000000053e7a3 in
cmCTestRunTest::EndTest(unsigned long, unsigned long, bool) ()
http://public.kitware.com/Bug/view.php?id=8  0x0000000000530e3b in
cmCTestMultiProcessHandler::CheckOutput() ()
http://public.kitware.com/Bug/view.php?id=9  0x00000000005340a8 in
cmCTestMultiProcessHandler::RunTests() ()
http://public.kitware.com/Bug/view.php?id=10 0x00000000004f7f2b in
cmCTestTestHandler::ProcessDirectory(std::vector<cmsys::String,
std::allocator<cmsys::String> >&, std::vector<cmsys::String,
std::allocator<cmsys::String> >&) ()
http://public.kitware.com/Bug/view.php?id=11 0x00000000004f82fb in
cmCTestTestHandler::ProcessHandler() ()
http://public.kitware.com/Bug/view.php?id=12 0x0000000000495e9e in
cmCTest::ProcessTests() ()
http://public.kitware.com/Bug/view.php?id=13 0x0000000000498652 in
cmCTest::Run(std::vector<std::string, std::allocator<std::string> >&,
std::string*) ()
http://public.kitware.com/Bug/view.php?id=14 0x0000000000486bdf in main ()


The problem came from : int res = cmCTest::HTTPRequest(url, cmCTest::HTTP_GET,
response, "", "", 3); in GetCDashVersion().

When not using CDash on projects I only use "enable_testing()" in CMakeLists.txt
so didn't set CDashDropSite (maybe this is a mistake, but what to setup if I
didn't intend to use cdash ???).

In such case, DropSite is empty, so requested url is
"http:///api/getversion.php" which lead to "http:" as domain lookup into curl.
It finish by waiting the 3 second before timout ... for nothing.

Partialy related to this commit : 41fcf6e1558cb3ef4639928f87d58531b45fe237.
Maybe another one interfer with it by setting an empty drop site by default as I
observe this bug for a couple of months.

Also observed on a Gentoo, ubuntu x86_64.

Steps to Reproduce: 
Setup a project with fast tests and only setup "enable_tesing()" in
CMakeLists.txt. Do not consider usage of cdash and simply compile and run tests
:

mkdir build
cd build
cmake ..
make
make test

You will wait at least 3 seconds.

Additional Information: 
Suggest to simply add of if statement before starting the request and bypass
this when drop site is empty.
====================================================================== 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2011-12-10 16:58 Sébastien ValatNew Issue                                    
======================================================================