OpenXML escaping illegal characters

Question

I am doing some string replacement within a Word Docx file using OpenXML Power Tools and it is working as expected. However things break when I have invalid characters in the substitution such as ampersand, so for instance "Harry & Sally" will break and produce an invalid document. According to this post illegal characters need to be converted to xHHHH.

I am having trouble finding the contents to the OOXML clause mentioned in the post and hence escaping characters appropriately.

I am hoping someone either has some code or insights into exactly what characters need to be escaped. I was also hopeful OpenXML Power Tools could do this for me in some way, but I cannot seem to find anything in there either.

Answer 1

The specification is just talking about the standard set of characters that have to be escaped in XML. The XML specification mentioned in the linked post is the one from the W3C, found here.

There are five characters that need to be escaped anywhere they appear in XML data (names, values, etc) unless they are part of a CDATA section. According to Section 2.4:

The ampersand character (&) and the left angle bracket (<) must not appear in their literal form, except when used as markup delimiters, or within a comment, a processing instruction, or a CDATA section. If they are needed elsewhere, they must be escaped using either numeric character references or the strings " & " and " < " respectively. The right angle bracket (>) may be represented using the string " > ", and must, for compatibility, be escaped using either " > " or a character reference when it appears in the string " ]]> " in content, when that string is not marking the end of a CDATA section.

To allow attribute values to contain both single and double quotes, the apostrophe or single-quote character (') may be represented as " ' ", and the double-quote character (") as " " ".

In other words, escape the following characters:

' -> '
" -> "
> -> >
< -> &lt;
& -> &amp;

Typically, you wouldn't encode these as xHHHH, you'd use the XML entities listed above, but either is allowed. You also don't need to encode quotes or the right-angle bracket in every case, only when they would otherwise represent XML syntax, but it's usually safer to do it all the time.

The XML specification also includes the list of every Unicode character that can appear in an XML document, in section 2.2:

Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]

That list includes basically every Unicode character in the Basic plane (every one you're likely to run into), except for the control characters. Only the tab, CR, and LF characters are allowed -- any other character below ASCII 32 (space) needs to be escaped.

The big gap in the list (0xD800-0xDFF) is for surrogate encoding values, which shouldn't appear by themselves anyway, as they're not valid characters. The last two, 0xFFFE and 0xFFFF, are also not valid characters.