Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Java InputStream encoding/charset

Running the following (example) code

import java.io.*;

public class test {
    public static void main(String[] args) throws Exception {
        byte[] buf = {-27};
        InputStream is = new ByteArrayInputStream(buf);
        BufferedReader r = new BufferedReader(
                new InputStreamReader(is, "ISO-8859-1"));
        String s = r.readLine();
        System.out.println("test.java:9 [byte] (char)" + (char)s.getBytes()[0] + 
                " (int)" + (int)s.getBytes()[0]);
        System.out.println("test.java:10 [char] (char)" + (char)s.charAt(0) + 
                " (int)" + (int)s.charAt(0));
        System.out.println("test.java:11 string below");
        System.out.println(s);
        System.out.println("test.java:13 string above");
    }
}

gives me this output

test.java:9 [byte] (char)? (int)63
test.java:10 [char] (char)? (int)229
test.java:11 string below
?
test.java:13 string above

How do I retain the correct byte value (-27) in the line-9 printout? And consequently receive the expected output of the System.out.println(s) command (å).

like image 625
Tobbe Avatar asked Jun 15 '10 08:06

Tobbe


People also ask

How do I change character encoding in Java?

java -Dfile. encoding="UTF-8" HelloWorld, we can specify UTF-8 charset. Method 2: Specifying the environment variable “JAVA_TOOLS_OPTIONS.” In case we start JVM starts up using some scripts and tools, the default charset can be set using the environment variable JAVA_TOOL_OPTIONS to -Dfile.

What is the default charset for Java?

The native character encoding of the Java programming language is UTF-16.

How does InputStreamReader work in Java?

An InputStreamReader is a bridge from byte streams to character streams: It reads bytes and decodes them into characters using a specified charset . The charset that it uses may be specified by name or may be given explicitly, or the platform's default charset may be accepted.


1 Answers

If you want to retain byte values, don't use a Reader at all, ideally. To represent arbitrary binary data in text and convert it back to binary data later, you should use base16 or base64 encoding.

However, to explain what's going on, when you call s.getBytes() that's using the default character encoding, which apparently doesn't include Unicode character U+00E5.

If you call s.getBytes("ISO-8859-1") everywhere instead of s.getBytes() I suspect you'll get back the right byte value... but relying on ISO-8859-1 for this is kinda dirty IMO.

like image 90
Jon Skeet Avatar answered Oct 03 '22 21:10

Jon Skeet