Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why charset names are not constants?

Charset issues are confusing and complicated by themselves, but on top of that you have to remember exact names of your charsets. Is it "utf8"? Or "utf-8"? Or maybe "UTF-8"? When searching internet for code samples you will see all of the above. Why not just make them named constants and use Charset.UTF8?

like image 860
serg Avatar asked Nov 05 '09 22:11

serg


People also ask

What is charset name?

Charset names generally follow the conventions documented in RFC 2278: IANA Charset Registration Procedures. Every charset has a canonical name and may also have one or more aliases. The canonical name is returned by the name method of this class. Canonical names are, by convention, usually in upper case.

What is Standardcharsets utf_8 in Java?

Introduction. When working with Strings in Java, we oftentimes need to encode them to a specific charset, such as UTF-8. UTF-8 represents a variable-width character encoding that uses between one and four eight-bit bytes to represent all valid Unicode code points.

What is encoding in Java?

Encoding is a way to convert data from one format to another. String objects use UTF-16 encoding. The problem with UTF-16 is that it cannot be modified. There is only one way that can be used to get different encoding i.e. byte[] array. The way of encoding is not suitable if we get unexpected data.


6 Answers

The simple answer to the question asked is that the available charset strings vary from platform to platform.

However, there are six that are required to be present, so constants could have been made for those long ago. I don't know why they weren't.

JDK 1.4 did a great thing by introducing the Charset type. At this point, they wouldn't have wanted to provide String constants anymore, since the goal is to get everyone using Charset instances. So why not provide the six standard Charset constants, then? I asked Martin Buchholz since he happens to be sitting right next to me, and he said there wasn't a really particularly great reason, except that at the time, things were still half-baked -- too few JDK APIs had been retrofitted to accept Charset, and of the ones that were, the Charset overloads usually performed slightly worse.

It's sad that it's only in JDK 1.6 that they finally finished outfitting everything with Charset overloads. And that this backwards performance situation still exists (the reason why is incredibly weird and I can't explain it, but is related to security!).

Long story short -- just define your own constants, or use Guava's Charsets class which Tony the Pony linked to (though that library is not really actually released yet).

Update: a StandardCharsets class is in JDK 7.

like image 97
Kevin Bourrillion Avatar answered Oct 06 '22 00:10

Kevin Bourrillion


Two years later, and Java 7's StandardCharsets now defines constants for the 6 standard charsets.

If you are stuck on Java 5/6, you can use Guava's Charsets constants, as suggested by Kevin Bourrillion and Jon Skeet.

like image 23
Etienne Neveu Avatar answered Oct 05 '22 23:10

Etienne Neveu


I'd argue that we can do much better than that... why aren't the guaranteed-to-be-available charsets accessible directly? Charset.UTF8 should be a reference to the Charset, not the name as a string. That way we wouldn't have to handle UnsupportedEncodingException all over the place.

Mind you, I also think that .NET chose a better strategy by defaulting to UTF-8 everywhere. It then screwed up by naming the "operating system default" encoding property simply Encoding.Default - which isn't the default within .NET itself :(

Back to ranting about Java's charset support - why isn't there a constructor for FileWriter/FileReader which takes a Charset? Basically those are almost useless classes due to that restriction - you almost always need an InputStreamReader around a FileInputStreamor the equivalent for output :(

Nurse, nurse - where's my medicine?

EDIT: It occurs to me that this hasn't really answered the question. The real answer is presumably either "nobody involved thought of it" or "somebody involved thought it was a bad idea." I would strongly suggest that in-house utility classes providing the names or charsets avoid duplication around the codebase... Or you could just use the one that we used at Google when this answer was first written. (Note that as of Java 7, you'd just use StandardCharsets instead.)

like image 41
Jon Skeet Avatar answered Oct 05 '22 23:10

Jon Skeet


In Java 1.7

import java.nio.charset.StandardCharsets

ex: StandardCharsets.UTF_8 StandardCharsets.US_ASCII

like image 39
Roger Avatar answered Oct 05 '22 23:10

Roger


The current state of the encoding API leaves something to be desired. Some parts of the Java 6 API don't accept Charset in place of a string (in logging, dom.ls, PrintStream; there may be others). It doesn't help that encodings are supposed to have different canonical names for different parts of the standard library.

I can understand how things got to where they are; not sure I have any brilliant ideas about how to fix them.


As an aside...

You can look up the names for Sun's Java 6 implementation here.

For UTF-8, the canonical values are "UTF-8" for java.nio and "UTF8" for java.lang and java.io. The only encodings the spec requires a JRE to support are: US-ASCII; ISO-8859-1; UTF-8; UTF-16BE; UTF-16LE; UTF-16.

like image 23
McDowell Avatar answered Oct 05 '22 22:10

McDowell


I have long ago defined a utility class with UTF_8, ISO_8859_1 and US_ASCII Charset constants.

Also, some long time ago ( 2+ years ) I did a simple performance test between new String( byte[], Charset ) and new String( byte[], String charset_name ) and discovered that the latter implementation is CONSIDERABLY faster. If you take a look under the hood at the source code you will see that they indeed follow quite a different path.

For that reason I included a utility in the same class

public static String stringFromByteArray (
    final byte[] array,
    final Charset charset
)
{
    try
    {
        return new String( array, charset.name( ) )
    }
    catch ( UnsupportedEncodingException ex )
    {
        // cannot happen
    }
}

Why the String( byte[], Charset ) constructor does not do the same, beats me.

like image 21
Alexander Pogrebnyak Avatar answered Oct 05 '22 23:10

Alexander Pogrebnyak