Menu
I download an HTML page. The HTTP content-type header specifies one character encoding, and the page has a meta tag that specifies another. What's the correct way to handle that?
I guess 'correct' isn't the right word, since nobody follows the damn standards anyway... so what's the way that will cause me the least problems?
458
One way to check this is to use the W3C Markup Validation Service. The validator usually detects the character encoding from the HTTP headers and information in the document. If the validator fails to detect the encoding, it can be selected on the validator result page via the 'Encoding' pulldown menu (example).
Always declare the encoding of your document using a meta element with a charset attribute, or using the http-equiv and content attributes (called a pragma directive).
The charset attribute specifies the character encoding for the HTML document. The HTML5 specification encourages web developers to use the UTF-8 character set, which covers almost all of the characters and symbols in the world!
Do the same as webbrowsers do: use the response header. When HTML is served over HTTP, the meta tag is ignored when the response header is present. Only when the HTML is read from local disk file system, the meta tag is been used. This is also explicitly specified by w3 HTML spec.
To sum up, conforming user agents must observe the following priorities when determining a document's character encoding (from highest priority to lowest):
- An HTTP "charset" parameter in a "Content-Type" field.
- A META declaration with "http-equiv" set to "Content-Type" and a value set for "charset".
- The charset attribute set on an element that designates an external resource.
Any existing decent HTML parser in whatever language you use should already take this into account. According your question history you're familiar with Java, I'd then suggest to grab Jsoup for this.
82
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
