[cmake-developers] RFC: pretty/HTML output in test logs in ctest/cdash

Alexander Neundorf neundorf at kde.org
Mon Apr 29 14:31:37 EDT 2013


On Thursday 18 April 2013, Alexander Neundorf wrote:
> Hi,
> 
> attached are two patches for ctest and cdash from Volkan (on CC).
> They are not ready for inclusion, but before we continue to work in this,
> we'd like to get some comments whether this is a good idea, the right
> approach, etc.
> So, here we go.
> 
> We are using cdash a lot, and for many of our tests the log is quite
> verbose and somewhat hard to read. When looking for the reason why a test
> failed, it can be somewhat hard to find the line which contains "ERROR".
> Also, a lot of stuff is repeated every line, which also makes the log
> harder to read, e.g. the directory part of the file where the log came
> from:
> 
> <timestamp> /some/where/deep/in/a/subdir/test.py: Something...
> <timestamp> /some/where/deep/in/a/subdir/test.py: happend...
> <timestamp> /some/where/deep/in/a/subdir/test.py: here...
> <timestamp> /some/where/deep/in/a/subdir/test.py: ...
> 
> The directory part is actually so wide that we have to scroll to see actual
> text.
> 
> 
> So, to help with this, we came up with the idea to use HTML to format the
> log so that it becomes easier to read.
> 
> The attached patch for ctest adds a new test property GENERATE_HTML, and if
> this is enabled, ctest generates HTML. It searches the lines which made the
> test fail and colors them red, and the lines which make a test pass are
> colored green.
> This makes it easier to find the fail/succeed lines.
> 
> (In the patch the fail strings are still hardcoded, this would have to use
> the respective test property.)
> 
> On the cdash side, if "</span>" is found in the log, it is considered HTML
> and not escaped. I guess this should be done in a different way, which
> does not involve parsing the log text.
> 
> Comments so far ?
> 
> 
> For making our log lines shorter, we are not sure yet how to do that.
> We could add support for squish XML to ctest, so that ctest reads the
> squish XML and converts it to HTML which is in some way shorter, maybe a
> HTML table, or something.
> Or we could add a test property with a regular expression, if that regexp
> matches, ctest replaces it with a shorter version or something.
> Or we could modify our test scripts so that they already generate the
> pretty HTML, which ctest then simply sends to cdash.
> That's the ideas we have so far, but it doesn't feel quite right yet.
> 
> So, what do you think ?

Any comments ?

Alex



More information about the cmake-developers mailing list