More unicode characters in windows console than expected

Question

I want to print russian and german characters in windows console. So I wrote a small test program to understand how well it works:

PrintStream ps = new PrintStream(System.out, false, "UTF-8");
ps.println("öäüß гджщ");

Then I started cmd.exe, changed its font to Lucida Console, which supports Unicode, changed code page to Unicode with "chcp 65001" and executed my program.

The german and russian characters were printed but there was a little more text than I expected (underlined with red):

But the text is printed correctly in the Eclipse console. Is there a way to print it correctly in windows console? I use Windows 7.

I've just solved the problem with JNI, but it is still interesting whether it is doable with pure java.

Answer 1

Every time you open or write a file, a certain encoding will be applied. But sometimes we forget that also our IDE (Eclipse in your case) has an encoding.

When you are typing a certain text between quotes, it is displayed and typed in a certain encoding, the encoding of your IDE. Your assumption is that the encoding of your output stream (UTF-8) will also guarantee that the text is displayed with that specific encoding. However, I think also here again the encoding of your IDE is applied.

I would propose to double check your encoding of eclipse. Perhaps this can solve your problem. Certainly worth a try, isn't it ? :)

For a global encoding setting add the following code to the eclipse.ini file

-Dfile.encoding=UTF-8 

EDIT:

I would just like to add the following. I performed the following steps as an experiment.

1. I opened Notepad++ and created a new file
2. I modified the encoding setting to UTF-8
3. I copied your Russian text and pasted it in my new text file and saved it.
4. Next I opened my windows console ("cmd")
5. I executed the "chcp 65001" command.
6. Next I printed the content of the file in my console: "type file.txt"
7. Everything shows correctly.

This does not confirm much, but it does confirm the fact that DOS can do the job if the content is foreseen in the right encoding.

EDIT2:

@ka3ak It's been over 2 years, but while reading a book about Java I/O I stumbled upon the following.

System.console().printf(...) has better support for special characters than the System.out.println(...) method.

Since the PrintStream just wraps around the System.out stream, I guess you have the same limitations. I am wondering if this could have solved the problem. If it still matters, please give it a try. :)

Other posts on stackoverflow report similar things: console.writeline and System.out.println