How to convert octal char sequence to unicode in Java

Question

Hi Have following string,

Let\342\200\231s start with the most obvious question first. This is what an \342\200\234unfurl\342\200\235 is

It is supposed to be displayed as The first three numbers (\342\200\231) actually represent a octal sequence http://graphemica.com/%E2%80%99 and its unicode equivalent is \u2019

Similarly \342\200\234 represents a octal sequence http://graphemica.com/%E2%80%9C and its unicode equivalent is \u201C

Is there any library or function which I can use to convert these octal sequences to their unicode equivalent?

Answer 1

The bytes you show are (a representation of) UTF-8 encoding, which is only one of many forms of Unicode. Java is designed to handle such encodings as byte sequences (such as arrays, and also streams), but not as chars and Strings. The somewhat cleaner way is to actually use bytes, but then you have to deal with the fact that Java bytes are signed (-128 .. +127) and all multibyte UTF-8 codes are (by design) in the upper half of 8-bit space:

byte[] a = {'L','e','t',(byte)0342,(byte)0200,(byte)0231,'s'};
System.out.println (new String (a,StandardCharsets.UTF_8));
// or arguably uglier
byte[] b = {'L','e','t',0342-256,0200-256,0231-256,'s'};
System.out.println (new String (b,StandardCharsets.UTF_8));

But if you want something closer to your original you can cheat just a little by treating a String (of unsigned chars) that actually contains the UTF-8 bytes as if it contained the 8-bit characters that form Unicode range 0000-00FF which is defined to be the same as ISO-8859-1:

byte[] c = "Let\342\200\231s".getBytes(StandardCharsets.ISO_8859_1);
System.out.println (new String (c,StandardCharsets.UTF_8));