MantisBT - CMake
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0008647CMakeCTestpublic2009-03-01 22:222009-03-02 16:31
ReporterRoscoe A. Bartlett 
Assigned ToBrad King 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0008647: CTest Dev: CTest sumitting invalid XML to dashboard resulting in tests getting dropped
Description
I am creating an issue for this email chain describing the problem with CTest submitting invalid XML in some cases resulting in tests getting dropped from the dashboard.

 

-----Original Message-----
From: Bartlett, Roscoe A
Sent: Friday, February 27, 2009 9:20 PM
To: 'Julien Jomier'
Cc: David Cole; Bill Hoffman; Perschbacher, Brent M
Subject: RE: [CDash 0008616]: Sending subproject email address regression lists in XML and updating CDash email lists

Hello guys,

I just realized what happened. I added new tests to Teuchos last night and I just noticed that the results from last night only show one Teuchos test. For example, if you look at:

    http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilinos&date=2009-02-28&display=project [^]

and look at the 'Nightly' test row:

    godel.sandia.gov Linux-Nightly-MPI_OPT 0 0 0 0 0.1 0 0 1.7 0 0 1 0 2009-02-27T00:14:05 MST Teuchos
 
you see Teuchos with just one test. This is before I updated CMake/Ctest or Cdash for at least several days. The night before showed 44 Teuchos tests that proves the bug was already in Ctest and/or Cdash before the upgrade today.

What aspect of the XML submission is invalid?

Anyway, it is not that Ctest or Cdash have changed to introduce a bug, it is that I added a new test that exposed an existing bug in Ctest and/or Cdash.

Thanks,

- Ross




> -----Original Message-----
> From: Julien Jomier [mailto:julien.jomier@kitware.com]
> Sent: Friday, February 27, 2009 8:35 PM
> To: Bartlett, Roscoe A
> Cc: David Cole; Bill Hoffman; Perschbacher, Brent M
> Subject: Re: [CDash 0008616]: Sending subproject email address
> regression lists in XML and updating CDash email lists
>
> There is an invalid character in the XML, that's probably why only one
> test is showing up. Dave, is this related to the changes to CTest? if
> the parsing fails, there is really nothing CDash can do...
>
> Julien
>
> Bartlett, Roscoe A wrote:
> > More info:
> >
> > The one test being displayed for Teuchos is the test
> Teuchos_BLAS_test which is the first test listed in the XML file.
> >
> > You can get one of the XML file that was submitted for Teuchos at:
> >
> >
> >
> http://www.cs.sandia.gov/~rabartl/stuff/Trilinos_godel.sandia.gov_expr [^]
> > -4_20090228-0055-Experimental_Test.xml
> >
> > Thanks,
> >
> > - Ross
> >
> >
> >
> >> -----Original Message-----
> >> From: Bartlett, Roscoe A
> >> Sent: Friday, February 27, 2009 6:11 PM
> >> To: 'Julien Jomier'
> >> Cc: 'David Cole'; 'Bill Hoffman'; Perschbacher, Brent M
> >> Subject: RE: [CDash 0008616]: Sending subproject email address
> >> regression lists in XML and updating CDash email lists
> >>
> >> More info on the problem:
> >>
> >> I submitted results for more Trilinos packages to:
> >>
> >>
> >> http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilino [^]
> >> s&date=2009-02-28&display=project
> >>
> >> with the new Ctest program and it reports the correct number of
> >> passing tests for RTOp and Thyra for the 'expr-4'
> >> experimental build. However, it is still showing just 1 test for
> >> Teuchos even through everything else (the console output
> from ctest
> >> on the client, the XML file sent etc) all show 49 tests.
> >>
> >> Again, let me know what you need to see.
> >>
> >> Thanks,
> >>
> >> - Ross
> >>
> >>
> >>
> >>
> >>
> >>> -----Original Message-----
> >>> From: Bartlett, Roscoe A
> >>> Sent: Friday, February 27, 2009 6:03 PM
> >>> To: 'Julien Jomier'
> >>> Cc: 'David Cole'; 'Bill Hoffman'; Perschbacher, Brent M
> >>> Subject: RE: [CDash 0008616]: Sending subproject email address
> >>> regression lists in XML and updating CDash email lists
> >>>
> >>> Julien,
> >>>
> >>> Okay, this must be some problem with Cdash because I used
> >> an old Ctest
> >>> from 2/16 and it is still just reporting 1 Teuchos test
> >> even through
> >>> 49 tests were run and submitted.
> >>>
> >>> What specific files or other information do you need to
> track this
> >>> down?
> >>>
> >>> Thanks,
> >>>
> >>> - Ross
> >>>
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Bartlett, Roscoe A
> >>>> Sent: Friday, February 27, 2009 5:52 PM
> >>>> To: 'Julien Jomier'
> >>>> Cc: David Cole; 'Bill Hoffman'; Perschbacher, Brent M
> >>>> Subject: RE: [CDash 0008616]: Sending subproject email address
> >>>> regression lists in XML and updating CDash email lists
> >>>>
> >>>> Julien,
> >>>>
> >>>> I did the update and it is not submitting the tests
> >>> correctly anymore.
> >>>> For example, if you look at:
> >>>>
> >>>>
> >>>> http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilino [^]
> >>>> s&date=2009-02-28&display=project
> >>>>
> >>>> and at expr-3 you see that it only shows 1 test for Teuchos even
> >>>> through it shows a bunch of tests run. I looked and the
> >>> full submit
> >>>> file:
> >>>>
> >>>>
> >>>>
> >>
> Trilinos_godel.sandia.gov_expr-3_20090228-0044-Experimental_Test.xml
> >>>> seems to be in the backup directory.
> >>>>
> >>>> I upgraded both Ctest and Cdash so I don't know where the
> >>> problem is.
> >>>> I think I have a older version of Ctest on another machine
> >>> I can try
> >>>> with the submit real fast.
> >>>>
> >>>> - Ross
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0010003closed Julien Jomier CDash Wrong escaping of utf-8 
related to 0009944closed David Cole CMake Wrong umlaut handling for Update.xml 
Attached Filesxml Trilinos_godel.sandia.gov_expr-4_20090228-0055-Experimental_Test.xml (334,477) 2009-03-02 09:19
https://public.kitware.com/Bug/file/2082/Trilinos_godel.sandia.gov_expr-4_20090228-0055-Experimental_Test.xml
Issue History
Date ModifiedUsernameFieldChange
2009-03-01 22:22Roscoe A. BartlettNew Issue
2009-03-02 09:15Bill HoffmanStatusnew => assigned
2009-03-02 09:15Bill HoffmanAssigned To => Brad King
2009-03-02 09:16Bill HoffmanNote Added: 0015484
2009-03-02 09:19Bill HoffmanFile Added: Trilinos_godel.sandia.gov_expr-4_20090228-0055-Experimental_Test.xml
2009-03-02 09:21Bill HoffmanNote Added: 0015485
2009-03-02 09:41Roscoe A. BartlettNote Added: 0015487
2009-03-02 09:58Brad KingNote Added: 0015488
2009-03-02 09:59Brad KingNote Added: 0015489
2009-03-02 10:01Brad KingNote Edited: 0015488
2009-03-02 12:08Roscoe A. BartlettNote Added: 0015492
2009-03-02 12:42Brad KingNote Added: 0015495
2009-03-02 13:00Bill HoffmanNote Added: 0015496
2009-03-02 16:30Brad KingNote Added: 0015501
2009-03-02 16:31Brad KingStatusassigned => closed
2009-03-02 16:31Brad KingResolutionopen => fixed
2009-12-07 10:58Brad KingRelationship addedrelated to 0010003
2010-12-14 16:36David ColeRelationship addedrelated to 0009944

Notes
(0015484)
Bill Hoffman   
2009-03-02 09:16   
Here is the xml file that has bad characters in it:

   http://www.cs.sandia.gov/~rabartl/stuff/Trilinos_godel.sandia.gov_expr-4_20090228-0055-Experimental_Test.xml [^]
(0015485)
Bill Hoffman   
2009-03-02 09:21   
The xml file attached has the issue. Line number 227.
 myGlobalReducts[0] = &#x1; == as<Packet>(numProcs) = &#x1; : passed

The trouble looks like there is an &# for some reason...

Ross, what is the actual output for this test at this point:


15. DefaultMpiComm_LongInt_char_reduceAllAndScatter_1_UnitTest ...
 myGlobalReducts[0] = &#x1; == as<Packet>(numProcs) = &#x1; : passed
 [Passed]
(0015487)
Roscoe A. Bartlett   
2009-03-02 09:41   
The output that I get sent to the console is:

 
15. DefaultMpiComm_LongInt_char_reduceAllAndScatter_1_UnitTest ...
 myGlobalReducts[0] = ^A == as<Packet>(numProcs) = ^A : passed
 [Passed]


That is very strange, why would '^A' get replaced by '&#x1;'?
(0015488)
Brad King   
2009-03-02 09:58   
(edited on: 2009-03-02 10:01)
Put this in your ctest script:

  set("ENV{LC_ALL}" C)

This will prevent gcc from outputting the non-standard chars.
(0015489)
Brad King   
2009-03-02 09:59   
This change will avoid the &#x1; syntax for now:

BUG: Hack for issue 0008647
/cvsroot/CMake/CMake/Source/cmXMLSafe.cxx,v <-- Source/cmXMLSafe.cxx
new revision: 1.4; previous revision: 1.3
(0015492)
Roscoe A. Bartlett   
2009-03-02 12:08   
I just did a CVS update of CMake (with CTest), built, and reran the Teuchos tests as an experimental build. All 49 tests seem to be submitting as shown at:

    http://trilinos-dev.sandia.gov/cdash/index.php?project=Trilinos&display=project&date=#Experimental [^]

and

    http://trilinos-dev.sandia.gov/cdash/viewTest.php?onlypassed&buildid=2160 [^]

Note that the test output at:

    http://trilinos-dev.sandia.gov/cdash/testDetails.php?test=20007&build=2160 [^]

shows the test out as:



15. DefaultMpiComm_LongInt_char_reduceAllAndScatter_1_UnitTest ...
 myGlobalReducts[0] = [bad-char-1] == as<Packet>(numProcs) = [bad-char-1] : passed
 [Passed]



Showing '[bad-char-1]' instead of '^A' is just fine with me. If I want to see the real output, I can just build and run the test myself. Or, I can look at the LastTest*.log file to see what it is.

This is fixed as far as I am concerned so unless this comes up again this issue is closed from my point of view. If you want to fix this in a "better" way we just need to make sure that the submits still work.
(0015495)
Brad King   
2009-03-02 12:42   
Great, thanks for trying it. The current fix will prevent the total breakage you've seen, but will cause GCC's error/warning messages to get a bunch of [bad-char-##] blocks in them unless you do the LC_ALL=C environment thing. Let me know if you need more details on the environment variable.

The full fix is unfortunately much more involved. We need to use the 'iconv' library to actually interpret the character encoding of the output, convert it to utf-8 if it isn't already, and only produce the [bad-char-##] syntax for bytes that are not valid encodings.
(0015496)
Bill Hoffman   
2009-03-02 13:00   
So, is the old <#> better than [bad-char- ##]? It might look better with a GCC done without LC_ALL=C.
(0015501)
Brad King   
2009-03-02 16:30   
I've committed a better intermediate fix. Now we use a "<0xNN>" syntax for byte values which are not valid according to the XML spec, and still use "&#xNN;" for other characters. This will preserve utf-8 encoding while avoiding bad XML data.

BUG: Avoid encoding invalid XML chars in CTest
/cvsroot/CMake/CMake/Source/cmXMLSafe.cxx,v <-- Source/cmXMLSafe.cxx
new revision: 1.5; previous revision: 1.4