Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why are "control" characters illegal in XML 1.0?

There are a variety of characters that are not legally encodeable in XML 1.0, e.g. U+0007 ('bell') and U+001B ('escape'). Most of the interesting ones are non-whitespace 'control' characters.

It's clear from (e.g.) this question and others that it's the XML spec that's the issue -- but can anyone illuminate me as to why the XML spec forbids these characters?

It seems like it could have been required that they be encoded in escapes, e.g. as  and  respectively, but perhaps there's a practical reason that the characters were forbidden rather than required to be escaped?

Answerers have suggested that there is some motivation towards avoiding transmission control characters, but Unicode includes many other control-like characters (consider U+200C "zero width non joiner"). I recognize there may be no good reason for this behavior, but I would still like to understand it better.

It's particularly frustrating because when those character values appear in other encodings data formats, I end up "double-escaping" new XML documents that need to encode this.

like image 717
Trochee Avatar asked Dec 31 '08 21:12

Trochee


People also ask

Are control characters allowed in XML?

Therefore, XML 1.1 allows the use of character references to the control characters #x1 through #x1F, most of which are forbidden in XML 1.0. For reasons of robustness, however, these characters still cannot be used directly in documents.

What is an illegal XML character?

The only illegal characters are & , < and > (as well as " or ' in attributes, depending on which character is used to delimit the attribute value: attr="must use &quot; here, ' is allowed" and attr='must use &apos; here, " is allowed' ).

How do you control characters in XML?

In XML 1.1, if you need to represent a control code explicitly the simplest alternative is to use an NCR (numeric character reference). For example, the control code ESC (Escape) U+001B would be represented by either the &#x1B; (hexadecimal) or &#27; (decimal) Numeric Character References.

Which characters are allowed in XML?

XML 1.0. Unicode code points in the following ranges are valid in XML 1.0 documents: U+0009, U+000A, U+000D: these are the only C0 controls accepted in XML 1.0; U+0020–U+D7FF, U+E000–U+FFFD: this excludes some (not all) non-characters in the BMP (all surrogates, U+FFFE and U+FFFF are forbidden);


2 Answers

My understanding is that this range is barred on the grounds that a markup language should not have any need to support transmission and flow control characters and including them would create a problem for any editors and parsers in binary conversion.

I'm struggling to find anything ex cathedra on this from Tim Bray et al though.

edit: some discussion of control chars and a vague admission it wasn't exactly over-engineered:

At 09:27 AM 17/06/00 -0500, Mark Volkmann wrote:

I've never seen a discussion of the reason why most ASCII control characters, such as a form feed, are not allowed in XML documents. Can anyone tell me the reason behind that decision or point me to a spec. that explains that?

I'm not sure we'd do it the same way if we were doing it again. I don't see that they do any real harm. Clearly, if you're optimizing for a highly interoperable content markup language (and XML is) it's legitimate to be suspicious of things like vertical-tab and backspace and so on... but then how can it be consistent to leave in \n and DEL and so on? -Tim

like image 170
annakata Avatar answered Oct 11 '22 16:10

annakata


It seems like it could have been required that they be encoded in escapes, e.g. as &#x0007; and &#x001B;

You can do exactly that in XML 1.1, for all but \0.

like image 30
bobince Avatar answered Oct 11 '22 16:10

bobince