Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Firefox and UTF-16 encoding

I'm building a website with the encoding UTF-16. It means that every files (html,jsp) is encoded in UTF-18 and I set in the head of every HTML page :

<meta http-equiv="content-type" content="text/html; charset=UTF-16">

My index page is correctly displayed by Chrom and IE. However, firefox doesn't render the index. It displays 2 strange characters and the full index page code :

��<!DOCTYPE html> <html> <head> <meta http-equiv="content-type" content="text/html; charset=UTF-16"> ...

Do you know the reason? It should be a problem of encoding, but I don't know where it's located...

Thanks

like image 327
user376112 Avatar asked Nov 29 '22 17:11

user376112


1 Answers

(Disclosure: I’m the developer responsible for the relevant code in Firefox.)

I'm building a website with the encoding UTF-16.

Please don’t. The short rules are:

  1. Never use UTF-16 for interchange.
  2. Always use UTF-8 for interchange.
  3. If you break rules 1 & 2 and still use UTF-16, at least use the BOM (the right one).
  4. But seriously, don’t break rules 1 and 2.

If you include user-provided content on your pages, using UTF-16 means that your site is vulnerable to socially engineered XSS at least in older browsers. Try this demo in an old version of Firefox (version 20 or older) or in a Presto-based version of Opera.

To avoid the vulnerability, use UTF-8.

It means that every files (html,jsp) is encoded in UTF-18

Uh oh. :-)

and I set in the head of every HTML page : <meta http-equiv="content-type" content="text/html; charset=UTF-16">

A meta tag works as an internal encoding declaration only when the encoding being used maps the bytes of the meta tag to the same bytes ASCII would. That’s not the case for UTF-16.

Do you know the reason?

Not without full response headers and the original response body in a hex editor. The general solution, as noted above, is to use always UTF-8 and never to use UTF-16 over HTTP.

If your content is in a language for which UTF-16 is more compact than UTF-8, two things:

  1. All the HTML, JS and CSS on the page is more compact in UTF-8.
  2. gzip makes the difference go away.
like image 60
hsivonen Avatar answered Dec 04 '22 05:12

hsivonen