View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008670CDashpublic2009-03-04 09:172009-03-30 08:18
ReporterRoscoe A. Bartlett 
Assigned ToJulien Jomier 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version1.4 
Summary0008670: CDash Dev: Failed tests shown in orange instead of red
Description
Why is it that failed tests are being shown in orange (like a warning)
instead of red (as an error) on the dashboard. For example, at:

  http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilinos&display=project&date=2009-03-05 [^]

the row:

  godel.sandia.gov Linux-Nightly-SERIAL_DEBUG_ICPC 0 0 0 0 0.2 0 46 0.7 0 1 49 0.1 2009-03-04T03:45:20 MST Teuchos

colors the '1' failed test for Teuchos in orange instead of red. I
assume this is not intended and is a defect since even one failed test
on other web pages is shown in read. For exmaple, on the page:


   http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilinos&date=2009-03-05 [^]

the row:

   Triutils 0 0 6 0 6 0 0 1 5 2009-03-04 10:58:52

shows the '1' failed test in red instead of orange.

This is not a high priority issue for Trilinos but it would help to
have all failed tests shown in red when scaning a large number of
builds as shown, for example, at:

  http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilinos&display=project&date=2009-03-05 [^]
TagsNo tags attached.
Attached Files

 Relationships

  Notes
(0015531)
Bill Hoffman (manager)
2009-03-04 09:31

 This is by design. I wonder if it can be made a project option.
(0015589)
Roscoe A. Bartlett (reporter)
2009-03-06 09:31

 I would be helpful if CDash would allow us to mark all errors in red.

Just and an observation, it seems that CDash is set up by default to encourage less than strong software engineering practices (e.g. emails only when an error first appears, coloring tests failures in orange). This is a system that encourages ignoring recurring errors.

In my opinion, CDash should be set up by default to encourage strong software engineering best practices that include, for example:

1) Most all failures are intolerable and 'stop the line' should be used to fix them ASAP (See "Implementing Lean Software Development" and "Continuous Integration" at http://www.cs.sandia.gov/~rabartl/readingList.html [^]).

2) If some error is 'tolerable', then that test and/or feature should disabled or removed and an appropriate bug item so some other event should occur to mark this. If the feature is still useful, then the issue should be pushed back onto the project's Sprint or Product Backlog to be addressed at a later time.

If other projects want to relax strong software engineering practices, then CDash should allow this through project options but these should not be the default, in my opinion.

Anyway, for Trilinos, all errors on supported builds are interoperable and they need to be dealt with in a timely manner.

Note that right now there are several errors on the Trilinos CDash website which I am allowing because it is useful to test and demonstrate the CTest/CDash software. Once we switch over to CTest/CDash as our official testing system, these will be dealt with and we will be all 'green' most days.
(0015827)
Julien Jomier (manager)
2009-03-27 23:29

 Test 'not run' are now showing up in yellow and test 'failed' showing up in red.

 Issue History
Date Modified Username Field Change
2009-03-04 09:17 Roscoe A. Bartlett New Issue
2009-03-04 09:30 Bill Hoffman Status new => assigned
2009-03-04 09:30 Bill Hoffman Assigned To => Julien Jomier
2009-03-04 09:31 Bill Hoffman Note Added: 0015531
2009-03-06 09:31 Roscoe A. Bartlett Note Added: 0015589
2009-03-27 23:29 Julien Jomier Note Added: 0015827
2009-03-27 23:29 Julien Jomier Status assigned => resolved
2009-03-27 23:29 Julien Jomier Fixed in Version => 1.4
2009-03-27 23:29 Julien Jomier Resolution open => fixed
2009-03-30 08:18 Julien Jomier Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team