[CMake] How to add a test and compare its output to a reference file ?

[Bill Hoffman]

Judicaël Bedouet wrote:
> Hi,
> 
> I 'm converting a project to CMake. Tests in this project print messages 
> to cout and these output traces are compared to reference files. If 
> output and its reference are different, the corresponding test is 
> considered as failed. I can't see how to make these tests work with CMake.
> 
> I try to run a complex command in ADD_TEST like
> ADD_TEST (Test test --arg_test > output && ${DIFF_COMMAND} output 
> reference_file).
> DIFF_COMMAND is equal to diff under Unix and fc under Windows. But CMake 
> interprets test commands and '>' is passed as an argument to test 
> ("test" "--arg_test" ">" "output" "&&" "diff" "output" "reference_file").
> 
> I try to set property PASS_REGULAR_EXPRESSION to contents of reference 
> file. It works for little files but it's difficult for large files :
>   - All characters * . ( ) ^ $ ... must be backslashed.
>   - In this project, generated files are encoded in UTF-8 and I'm not 
> sure that CMake regular expressions support UTF-8.
>   - When there is a difference, I can't see where it is. I understand 
> that PASS_REGULAR_EXPRESSION wasn't thought in terms of matching whole file.
> 
> I could modify source of program tests by redirecting std::cout to a 
> file and comparing this file to the reference file. But before doing 
> this, I would like to know if there is a way to make a test with command 
> like `test args > file && diff` or if it's possible to add a test 
> property which compare output of the program to a no regular expression.
> 

You can do something like this:

ADD_TEST(MyTestCreate Test test --arg_test --output output)
ADD_TEST(MyTestCreateCompare ${CMAKE_COMMAND} -E compare_files output 
outputTest)


-Bill