MantisBT - CDash |
| View Issue Details |
|
| ID | Project | Category | View Status | Date Submitted | Last Update |
| 0010332 | CDash | | public | 2010-02-25 14:45 | 2010-04-16 13:36 |
|
| Reporter | Roman Shtylman | |
| Assigned To | Julien Jomier | |
| Priority | normal | Severity | major | Reproducibility | always |
| Status | closed | Resolution | no change required | |
| Platform | | OS | | OS Version | |
| Product Version | 1.6.4 | |
| Target Version | | Fixed in Version | 1.6.4 | |
|
| Summary | 0010332: cdash does not properly handle large xml files |
| Description | If my project has lots of tests and produces lots of output because of it, cdash will fail to handle the entire xml file. The problem lies in the fact that the ctestparser.php file only does "$content = fread($filehandle, 8192);" (around line 44) once. This means that large files are truncated.
This is a regression from previous versions. |
| Steps To Reproduce | |
| Additional Information | This is easily fixed by adding the following under that line:
while (!feof($filehandler))
$content .= fread($filehandler, 8192);
likewise just using the above should work as well without the first $content read since the while loop will handle it but I have not tested that specific case after I did the initial fix.
|
| Tags | No tags attached. |
| Relationships | |
| Attached Files |  Test.xml (9,929) 2010-02-25 15:22
https://public.kitware.com/Bug/file/2902/Test.xml |
|
| Issue History |
| Date Modified | Username | Field | Change |
| 2010-02-25 14:45 | Roman Shtylman | New Issue | |
| 2010-02-25 14:46 | Julien Jomier | Status | new => assigned |
| 2010-02-25 14:46 | Julien Jomier | Assigned To | => Julien Jomier |
| 2010-02-25 14:50 | Julien Jomier | Note Added: 0019621 | |
| 2010-02-25 15:05 | Roman Shtylman | Note Added: 0019622 | |
| 2010-02-25 15:09 | Julien Jomier | Note Added: 0019623 | |
| 2010-02-25 15:22 | Roman Shtylman | File Added: Test.xml | |
| 2010-02-25 15:24 | Roman Shtylman | Note Added: 0019624 | |
| 2010-02-25 15:39 | Julien Jomier | Note Added: 0019627 | |
| 2010-02-25 15:57 | Roman Shtylman | Note Added: 0019628 | |
| 2010-02-25 16:02 | Julien Jomier | Note Added: 0019629 | |
| 2010-02-25 16:03 | Roman Shtylman | Note Added: 0019630 | |
| 2010-02-25 16:09 | Julien Jomier | Note Added: 0019631 | |
| 2010-02-25 16:15 | Roman Shtylman | Note Added: 0019632 | |
| 2010-02-25 18:47 | Roman Shtylman | Note Added: 0019637 | |
| 2010-02-25 22:31 | Julien Jomier | Status | assigned => resolved |
| 2010-02-25 22:31 | Julien Jomier | Fixed in Version | => 1.6.4 |
| 2010-02-25 22:31 | Julien Jomier | Resolution | open => no change required |
| 2010-04-16 13:36 | Julien Jomier | Status | resolved => closed |
|
Notes |
|
|
(0019621)
|
|
Julien Jomier
|
|
2010-02-25 14:50
|
|
|
The while loop is setup line 202. Are you sure this is the issue? We have been dealing with large XML file without any issues. Which error are you getting? |
|
|
|
(0019622)
|
|
Roman Shtylman
|
|
2010-02-25 15:05
|
|
From what I can see the file is parsed by xml_parse($parser, $content, false) on line 0000070:0000115. The problem I was having was that my test reports where not being submitted. I tried submitting them manually using curl (as well as usual ctest methods) and no error is returned. It adds the entry for the build but the test columns are blank. If I add my fix, and redo the curl submit the columns are populated as I expect.
The loop on 202 seems to happen after the file is parsed... I might be missing something there tho. What I do know is that the above fixed my problem :/ I can explore more to see what else happened if that is needed. |
|
|
|
(0019623)
|
|
Julien Jomier
|
|
2010-02-25 15:09
|
|
|
We use SAX parsing with parse the file by chunk. We parse it once to determine which XML handler should be allocated depending on the type of file (build, update, etc...). Could you attach your XML file so I can try it here and see if there is something wrong? |
|
|
|
(0019624)
|
|
Roman Shtylman
|
|
2010-02-25 15:24
|
|
|
I have attached a Test.xml file which causes cdash to fail for me. I can run ctest -D Nightly just fine. It also says upload successful, but when viewing the dashboard the test columns are all blank as if no tests were run. The database entries for tests are all -1. |
|
|
|
(0019627)
|
|
Julien Jomier
|
|
2010-02-25 15:39
|
|
|
This is strange. I just tested your file and it works well for me. I'm wondering if their is an issue with the current parser. |
|
|
|
(0019628)
|
|
Roman Shtylman
|
|
2010-02-25 15:57
|
|
|
That is interesting indeed. I even tested it against svn trunk ... with the same failed results. |
|
|
|
(0019629)
|
|
Julien Jomier
|
|
2010-02-25 16:02
|
|
|
I'm wondering if your PHP installation allows for streaming XML parsing, what kind of server (apache, PHP version, etc...) are you using? |
|
|
|
(0019630)
|
|
Roman Shtylman
|
|
2010-02-25 16:03
|
|
php5.2.12
on apache2.2.14
maybe its an option in the php.ini? The thing is I didn't have this problem with cdash 1.4 |
|
|
|
(0019631)
|
|
Julien Jomier
|
|
2010-02-25 16:09
|
|
|
Could you make sure that the backup directory is writeable by the apache/web server? |
|
|
|
(0019632)
|
|
Roman Shtylman
|
|
2010-02-25 16:15
|
|
|
It is writable and I even see the xml files in there. What is even more interesting is that I see the *full* xml file in there. |
|
|
|
(0019637)
|
|
Roman Shtylman
|
|
2010-02-25 18:47
|
|
|
stupid me... I had async upload turned on and this was causing some weird behavior... I disabled it and it seems to be fine now... sorry for the bug report. |
|