View Issue Details [ Jump to Notes ] | [ Print ] | ||||||||
ID | Project | Category | View Status | Date Submitted | Last Update | ||||
0007051 | CDash | public | 2008-05-19 12:03 | 2008-07-29 13:26 | |||||
Reporter | Brad King | ||||||||
Assigned To | Julien Jomier | ||||||||
Priority | normal | Severity | minor | Reproducibility | N/A | ||||
Status | closed | Resolution | fixed | ||||||
Platform | OS | OS Version | |||||||
Product Version | 1.2 | ||||||||
Target Version | Fixed in Version | 1.2 | |||||||
Summary | 0007051: Timing tests have too many false positives | ||||||||
Description | The algorithm to detect time failures needs to be revised. One time sample lying outside the majority of the data is *not* a failure. Single outliers can occur from busy machines or any number of reasons. Only several samples can define a failure. Statistically, the test would be something like deciding if the last N days are sampled from a distribution with a different mean than the distribution of the preceding M days (with M much bigger than N). Such a formal test would probably be too expensive. Instead we could use some kind of hack to reduce false positives by requiring that the last 3 or 4 days be outliers according to the current test before reporting the failure. | ||||||||
Tags | No tags attached. | ||||||||
Attached Files | |||||||||
Relationships | |
Relationships |
Notes | |
(0012861) Julien Jomier (manager) 2008-07-29 13:26 |
There is now a "Test time # max failures before flag" variable per project. That means that it doesn't display the timing error if the error occurs less than the max number of testing time failure. Thanks for the report. |
Notes |
Issue History | |||
Date Modified | Username | Field | Change |
2008-05-19 12:03 | Brad King | New Issue | |
2008-05-19 12:06 | Julien Jomier | Status | new => assigned |
2008-05-19 12:06 | Julien Jomier | Assigned To | => Julien Jomier |
2008-07-29 13:26 | Julien Jomier | Status | assigned => closed |
2008-07-29 13:26 | Julien Jomier | Note Added: 0012861 | |
2008-07-29 13:26 | Julien Jomier | Resolution | open => fixed |
2008-07-29 13:26 | Julien Jomier | Fixed in Version | => 1.2 |
Issue History |
Copyright © 2000 - 2018 MantisBT Team |