View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008647CMakeCTestpublic2009-03-01 22:222009-03-02 16:31
ReporterRoscoe A. Bartlett 
Assigned ToBrad King 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0008647: CTest Dev: CTest sumitting invalid XML to dashboard resulting in tests getting dropped
Description
I am creating an issue for this email chain describing the problem with CTest submitting invalid XML in some cases resulting in tests getting dropped from the dashboard.

 

-----Original Message-----
From: Bartlett, Roscoe A
Sent: Friday, February 27, 2009 9:20 PM
To: 'Julien Jomier'
Cc: David Cole; Bill Hoffman; Perschbacher, Brent M
Subject: RE: [CDash 0008616]: Sending subproject email address regression lists in XML and updating CDash email lists

Hello guys,

I just realized what happened. I added new tests to Teuchos last night and I just noticed that the results from last night only show one Teuchos test. For example, if you look at:

    http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilinos&date=2009-02-28&display=project [^]

and look at the 'Nightly' test row:

    godel.sandia.gov Linux-Nightly-MPI_OPT 0 0 0 0 0.1 0 0 1.7 0 0 1 0 2009-02-27T00:14:05 MST Teuchos
 
you see Teuchos with just one test. This is before I updated CMake/Ctest or Cdash for at least several days. The night before showed 44 Teuchos tests that proves the bug was already in Ctest and/or Cdash before the upgrade today.

What aspect of the XML submission is invalid?

Anyway, it is not that Ctest or Cdash have changed to introduce a bug, it is that I added a new test that exposed an existing bug in Ctest and/or Cdash.

Thanks,

- Ross




> -----Original Message-----
> From: Julien Jomier [mailto:julien.jomier@kitware.com]
> Sent: Friday, February 27, 2009 8:35 PM
> To: Bartlett, Roscoe A
> Cc: David Cole; Bill Hoffman; Perschbacher, Brent M
> Subject: Re: [CDash 0008616]: Sending subproject email address
> regression lists in XML and updating CDash email lists
>
> There is an invalid character in the XML, that's probably why only one
> test is showing up. Dave, is this related to the changes to CTest? if
> the parsing fails, there is really nothing CDash can do...
>
> Julien
>
> Bartlett, Roscoe A wrote:
> > More info:
> >
> > The one test being displayed for Teuchos is the test
> Teuchos_BLAS_test which is the first test listed in the XML file.
> >
> > You can get one of the XML file that was submitted for Teuchos at:
> >
> >
> >
> http://www.cs.sandia.gov/~rabartl/stuff/Trilinos_godel.sandia.gov_expr [^]
> > -4_20090228-0055-Experimental_Test.xml
> >
> > Thanks,
> >
> > - Ross
> >
> >
> >
> >> -----Original Message-----
> >> From: Bartlett, Roscoe A
> >> Sent: Friday, February 27, 2009 6:11 PM
> >> To: 'Julien Jomier'
> >> Cc: 'David Cole'; 'Bill Hoffman'; Perschbacher, Brent M
> >> Subject: RE: [CDash 0008616]: Sending subproject email address
> >> regression lists in XML and updating CDash email lists
> >>
> >> More info on the problem:
> >>
> >> I submitted results for more Trilinos packages to:
> >>
> >>
> >> http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilino [^]
> >> s&date=2009-02-28&display=project
> >>
> >> with the new Ctest program and it reports the correct number of
> >> passing tests for RTOp and Thyra for the 'expr-4'
> >> experimental build. However, it is still showing just 1 test for
> >> Teuchos even through everything else (the console output
> from ctest
> >> on the client, the XML file sent etc) all show 49 tests.
> >>
> >> Again, let me know what you need to see.
> >>
> >> Thanks,
> >>
> >> - Ross
> >>
> >>
> >>
> >>
> >>
> >>> -----Original Message-----
> >>> From: Bartlett, Roscoe A
> >>> Sent: Friday, February 27, 2009 6:03 PM
> >>> To: 'Julien Jomier'
> >>> Cc: 'David Cole'; 'Bill Hoffman'; Perschbacher, Brent M
> >>> Subject: RE: [CDash 0008616]: Sending subproject email address
> >>> regression lists in XML and updating CDash email lists
> >>>
> >>> Julien,
> >>>
> >>> Okay, this must be some problem with Cdash because I used
> >> an old Ctest
> >>> from 2/16 and it is still just reporting 1 Teuchos test
> >> even through
> >>> 49 tests were run and submitted.
> >>>
> >>> What specific files or other information do you need to
> track this
> >>> down?
> >>>
> >>> Thanks,
> >>>
> >>> - Ross
> >>>
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Bartlett, Roscoe A
> >>>> Sent: Friday, February 27, 2009 5:52 PM
> >>>> To: 'Julien Jomier'
> >>>> Cc: David Cole; 'Bill Hoffman'; Perschbacher, Brent M
> >>>> Subject: RE: [CDash 0008616]: Sending subproject email address
> >>>> regression lists in XML and updating CDash email lists
> >>>>
> >>>> Julien,
> >>>>
> >>>> I did the update and it is not submitting the tests
> >>> correctly anymore.
> >>>> For example, if you look at:
> >>>>
> >>>>
> >>>> http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilino [^]
> >>>> s&date=2009-02-28&display=project
> >>>>
> >>>> and at expr-3 you see that it only shows 1 test for Teuchos even
> >>>> through it shows a bunch of tests run. I looked and the
> >>> full submit
> >>>> file:
> >>>>
> >>>>
> >>>>
> >>
> Trilinos_godel.sandia.gov_expr-3_20090228-0044-Experimental_Test.xml
> >>>> seems to be in the backup directory.
> >>>>
> >>>> I upgraded both Ctest and Cdash so I don't know where the
> >>> problem is.
> >>>> I think I have a older version of Ctest on another machine
> >>> I can try
> >>>> with the submit real fast.
> >>>>
> >>>> - Ross
TagsNo tags attached.
Attached Filesxml file icon Trilinos_godel.sandia.gov_expr-4_20090228-0055-Experimental_Test.xml [^] (334,477 bytes) 2009-03-02 09:19

 Relationships
related to 0010003closedJulien Jomier CDash Wrong escaping of utf-8 
related to 0009944closedDavid Cole CMake Wrong umlaut handling for Update.xml 

  Notes
(0015484)
Bill Hoffman (manager)
2009-03-02 09:16

 Here is the xml file that has bad characters in it:

   http://www.cs.sandia.gov/~rabartl/stuff/Trilinos_godel.sandia.gov_expr-4_20090228-0055-Experimental_Test.xml [^]
(0015485)
Bill Hoffman (manager)
2009-03-02 09:21

 The xml file attached has the issue. Line number 227.
 myGlobalReducts[0] = &#x1; == as<Packet>(numProcs) = &#x1; : passed

The trouble looks like there is an &# for some reason...

Ross, what is the actual output for this test at this point:


15. DefaultMpiComm_LongInt_char_reduceAllAndScatter_1_UnitTest ...
 myGlobalReducts[0] = &#x1; == as<Packet>(numProcs) = &#x1; : passed
 [Passed]
(0015487)
Roscoe A. Bartlett (reporter)
2009-03-02 09:41

 The output that I get sent to the console is:

 
15. DefaultMpiComm_LongInt_char_reduceAllAndScatter_1_UnitTest ...
 myGlobalReducts[0] = ^A == as<Packet>(numProcs) = ^A : passed
 [Passed]


That is very strange, why would '^A' get replaced by '&#x1;'?
(0015488)
Brad King (manager)
2009-03-02 09:58
edited on: 2009-03-02 10:01

 Put this in your ctest script:

  set("ENV{LC_ALL}" C)

This will prevent gcc from outputting the non-standard chars.
(0015489)
Brad King (manager)
2009-03-02 09:59

 This change will avoid the &#x1; syntax for now:

BUG: Hack for issue 0008647
/cvsroot/CMake/CMake/Source/cmXMLSafe.cxx,v <-- Source/cmXMLSafe.cxx
new revision: 1.4; previous revision: 1.3
(0015492)
Roscoe A. Bartlett (reporter)
2009-03-02 12:08

 I just did a CVS update of CMake (with CTest), built, and reran the Teuchos tests as an experimental build. All 49 tests seem to be submitting as shown at:

    http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilinos&display=project&date=#Experimental [^]

and

    http://trilinos-dev.sandia.gov/cdash/viewTest.php?onlypassed&buildid=2160 [^]

Note that the test output at:

    http://trilinos-dev.sandia.gov/cdash/testDetails.php?test=20007&build=2160 [^]

shows the test out as:



15. DefaultMpiComm_LongInt_char_reduceAllAndScatter_1_UnitTest ...
 myGlobalReducts[0] = [bad-char-1] == as<Packet>(numProcs) = [bad-char-1] : passed
 [Passed]



Showing '[bad-char-1]' instead of '^A' is just fine with me. If I want to see the real output, I can just build and run the test myself. Or, I can look at the LastTest*.log file to see what it is.

This is fixed as far as I am concerned so unless this comes up again this issue is closed from my point of view. If you want to fix this in a "better" way we just need to make sure that the submits still work.
(0015495)
Brad King (manager)
2009-03-02 12:42

 Great, thanks for trying it. The current fix will prevent the total breakage you've seen, but will cause GCC's error/warning messages to get a bunch of [bad-char-##] blocks in them unless you do the LC_ALL=C environment thing. Let me know if you need more details on the environment variable.

The full fix is unfortunately much more involved. We need to use the 'iconv' library to actually interpret the character encoding of the output, convert it to utf-8 if it isn't already, and only produce the [bad-char-##] syntax for bytes that are not valid encodings.
(0015496)
Bill Hoffman (manager)
2009-03-02 13:00

 So, is the old <#> better than [bad-char- ##]? It might look better with a GCC done without LC_ALL=C.
(0015501)
Brad King (manager)
2009-03-02 16:30

 I've committed a better intermediate fix. Now we use a "<0xNN>" syntax for byte values which are not valid according to the XML spec, and still use "&#xNN;" for other characters. This will preserve utf-8 encoding while avoiding bad XML data.

BUG: Avoid encoding invalid XML chars in CTest
/cvsroot/CMake/CMake/Source/cmXMLSafe.cxx,v <-- Source/cmXMLSafe.cxx
new revision: 1.5; previous revision: 1.4

 Issue History
Date Modified Username Field Change
2009-03-01 22:22 Roscoe A. Bartlett New Issue
2009-03-02 09:15 Bill Hoffman Status new => assigned
2009-03-02 09:15 Bill Hoffman Assigned To => Brad King
2009-03-02 09:16 Bill Hoffman Note Added: 0015484
2009-03-02 09:19 Bill Hoffman File Added: Trilinos_godel.sandia.gov_expr-4_20090228-0055-Experimental_Test.xml
2009-03-02 09:21 Bill Hoffman Note Added: 0015485
2009-03-02 09:41 Roscoe A. Bartlett Note Added: 0015487
2009-03-02 09:58 Brad King Note Added: 0015488
2009-03-02 09:59 Brad King Note Added: 0015489
2009-03-02 10:01 Brad King Note Edited: 0015488
2009-03-02 12:08 Roscoe A. Bartlett Note Added: 0015492
2009-03-02 12:42 Brad King Note Added: 0015495
2009-03-02 13:00 Bill Hoffman Note Added: 0015496
2009-03-02 16:30 Brad King Note Added: 0015501
2009-03-02 16:31 Brad King Status assigned => closed
2009-03-02 16:31 Brad King Resolution open => fixed
2009-12-07 10:58 Brad King Relationship added related to 0010003
2010-12-14 16:36 David Cole Relationship added related to 0009944

Copyright © 2000 - 2018 MantisBT Team