[CMake] How to add a test and compare its output to a reference file ?

[Michael Wild]

On 12. Sep, 2008, at 16:14, Bill Hoffman wrote:

> Judicaël Bedouet wrote:
>> Hi,
>> I 'm converting a project to CMake. Tests in this project print  
>> messages to cout and these output traces are compared to reference  
>> files. If output and its reference are different, the corresponding  
>> test is considered as failed. I can't see how to make these tests  
>> work with CMake.
>> I try to run a complex command in ADD_TEST like
>> ADD_TEST (Test test --arg_test > output && ${DIFF_COMMAND} output  
>> reference_file).
>> DIFF_COMMAND is equal to diff under Unix and fc under Windows. But  
>> CMake interprets test commands and '>' is passed as an argument to  
>> test ("test" "--arg_test" ">" "output" "&&" "diff" "output"  
>> "reference_file").
>> I try to set property PASS_REGULAR_EXPRESSION to contents of  
>> reference file. It works for little files but it's difficult for  
>> large files :
>>  - All characters * . ( ) ^ $ ... must be backslashed.
>>  - In this project, generated files are encoded in UTF-8 and I'm  
>> not sure that CMake regular expressions support UTF-8.
>>  - When there is a difference, I can't see where it is. I  
>> understand that PASS_REGULAR_EXPRESSION wasn't thought in terms of  
>> matching whole file.
>> I could modify source of program tests by redirecting std::cout to  
>> a file and comparing this file to the reference file. But before  
>> doing this, I would like to know if there is a way to make a test  
>> with command like `test args > file && diff` or if it's possible to  
>> add a test property which compare output of the program to a no  
>> regular expression.
>
> You can do something like this:
>
> ADD_TEST(MyTestCreate Test test --arg_test --output output)
> ADD_TEST(MyTestCreateCompare ${CMAKE_COMMAND} -E compare_files  
> output outputTest)
>
>
> -Bill


Or, if you don't want to change your test-runners, you can create a  
runTest.cmake script which EXECUTE_PROCESSes the test-runner, puts the  
output into a file, captures exit status and stderr and then does a  
compare_files as Bill has shown above.

Find attached a not-so elegant proof of concept.


HTH

Michael

-------------- next part --------------
A non-text attachment was scrubbed...
Name: runTest.zip
Type: application/zip
Size: 1445 bytes
Desc: not available
URL: <http://www.cmake.org/pipermail/cmake/attachments/20080912/8917f8f4/attachment.zip>
-------------- next part --------------