[cmake-developers] [CMake 0013806]: list(SORT) produces unavoidable data corruption (likely root cause: improper semi-colon string *payload* handling in CMake)

[Mantis Bug Tracker]

The following issue has been SUBMITTED. 
====================================================================== 
http://www.itk.org/Bug/view.php?id=13806 
====================================================================== 
Reported By:                Andreas Mohr
Assigned To:                
====================================================================== 
Project:                    CMake
Issue ID:                   13806
Category:                   CMake
Reproducibility:            always
Severity:                   major
Priority:                   urgent
Status:                     new
====================================================================== 
Date Submitted:             2012-12-20 08:52 EST
Last Modified:              2012-12-20 08:52 EST
====================================================================== 
Summary:                    list(SORT) produces unavoidable data corruption
(likely root cause: improper semi-colon string *payload* handling in CMake)
Description: 
I just wanted to extend a test case for my work on a new ENVIRONMENT option to
add_custom_command() (adding many "interesting" env var key/value tests for
escaping of XML, batch, shell, ... specials),
and ended up realizing that any escaped semi-colon string content will end up
getting broken, hard, by CMake.

The test provided below will produce the following output:

$ cmake .
tlist pre-sort:
hi;there
cruel;world

tlist post-sort:
cruel
world
hi
there

extra_escaped_list pre-sort:
hi;there
cruel;world

extra_escaped_list post-sort:
cruel
world
hi
there

extra_escaped_list_i_mean_it pre-sort:
hi\\
there
cruel\\
world

extra_escaped_list_i_mean_it post-sort:
cruel\\
hi\\
there
world

-- Configuring done
-- Generating done



This shows that any SORT of the list will cause unsurviveable data corruption,
which is a major, INSURMOUNTABLE problem (as the futile attempts at
backslash-escaping in the test case below illustrate) when attempting to compile
a list of patterns for escape tests towards rather unrelated system components.

This wide-spread data corruption by CMake core layers should be fixed quickly
(we're now at 2.8.x, finally having reached good usability on the very large
number of systems that CMake supports, where I really wouldn't have expected
such data corruption issues to have remained).
I'd deem data corruption bugs to be of the near-highest level on the priority
scale (with perhaps actual security issues topping it), thereby assigning
priority urgent.
(I did read "[sldev] Semicolons and CMake"
https://lists.secondlife.com/pipermail/sldev/2009-April/013502.html , and have
to admit I walked away unconvinced)

This should possibly be handled by introducing a new case to the CMake policy
mechanism, to preserve the (reportedly quite important in this case) bug-for-bug
compat in older code.

If it isn't possible to fix this problem cleanly (or e.g. not in a single
evolution step), then one should think of other possibilities to be able to work
around the currently unavoidable data corruption. One way might be to introduce
a special CMAKE_ESCAPE_* variable which when inserted marks content in a special
manner to ensure proper handling.
Or possibly one could add new cmake_escape_*() function helpers rather than
resorting to often unclean global-variable-based handling.

Thanks!

Steps to Reproduce: 
cmake_minimum_required(VERSION 2.8)

project(list_escape_semicolon_test NONE)

function(show_list _title _list)
  message("${_title}:")
  foreach(elem_ ${_list})
    message("${elem_}")
  endforeach(elem_ ${_list})
  message("")
endfunction(show_list _title _list)

function(process_list _name _list)
  show_list("${_name} pre-sort" "${_list}")
  list(SORT _list)
  show_list("${_name} post-sort" "${_list}")
endfunction(process_list _name _list)

function(test_list_sort_escaping)
  set(tlist "")
  list(APPEND tlist "hi\;there")
  list(APPEND tlist "cruel\;world")
  process_list(tlist "${tlist}")
  set(tlist "")
  list(APPEND tlist "hi\\;there")
  list(APPEND tlist "cruel\\;world")
  process_list(extra_escaped_list "${tlist}")
  set(escape_string "\\\\")
  set(tlist "")
  list(APPEND tlist "hi${escape_string};there")
  list(APPEND tlist "cruel${escape_string};world")
  process_list(extra_escaped_list_i_mean_it "${tlist}")
endfunction(test_list_sort_escaping)

test_list_sort_escaping()


Additional Information: 
That's now the second (and unrelated) time in about two weeks that I stumbled
(and fell) over this (the first time being reading in a file(STRINGS) with
semi-colon payload and iterating over elements of the resulting list).
====================================================================== 

Issue History 
Date Modified    Username       Field                    Change               
====================================================================== 
2012-12-20 08:52 Andreas Mohr   New Issue                                    
======================================================================