Single Chinese character determined as length 2 in Java/Scala String

Question

I'm trying to split out all the Chinese characters from a String, but I bumped into a strange situation for the character 𥑮

scala> "𥑮"
res1: String = 𥑮

scala> res1.length
res2: Int = 2

scala> res1.getBytes
res3: Array[Byte] = Array(-16, -91, -111, -82)

scala> res1(0)
res4: Char = ?

scala> res1(1)
res5: Char = ?

It's a single character, but Java/Scala determine it as two unknown characters. And usually I see Chinese character taking three bytes in UTF-8, but this character takes four.

Hence, I can't split a String and find this single character. Even worse, when using myString.replaceAll("[^\\p{script=Han}]", "") to kick out all the non-Chinese characters, the second part of 𥑮 is replaced and it becomes an invalid String.

Is there any solution to this? I'm using openjdk-8-jdk on Ubuntu.

Answer 1

For length you should use

string.codePointCount(0, string.length());

For replacement it is best to avoid regex, which is char-based. You could write a loop relying on String#offsetByCodePoints() and manually remove characters based on String.codePointAt() and Character.isIdeographic().