.NET Localization: Japanese Characters Display as Squares

Question

OK, I don't do much .Net programming, but I do have one that I maintain, so the answer to this may be obvious.

Setup:

• Windows 7 Ultimate with All Language Packs Installed
• Visual Studio 2008 Winforms VB.Net project.

I'm in the process of localizing this project, and when I'm making the Japanese version of the forms, the characters display as squares, though they render in my browser correctly. I'm guessing that this is because the default font does not have a glyph for those characters.

So, my questions:

Are winforms UTF-8, or some other character encoding? Is there a way to change the character encoding? Should I change the font for the Japanese forms, or will Windows do it? What's the general best practice here?

I want to know that I am copying the characters correctly into my forms, and I want to be able to test them. How can I do this?

Thanks for any feedback!

EDIT: Thanks for the info! Here's what I've found.

Arial Unicode MS does have all the glyphs, but I wasn't using it, because it wasn't in the VS2008 list of fonts. I manually edited the font box to use it, but then Visual Studio throws the message, "Attempted to read or write protected memory. This is often an indication that other memory has been corrupted." I'm guessing that's because VS doesn't have permission to access that font for some reason. I go back to the default font, scary error message goes away.

Then, even when using Arial Unicode MS, the text renders as blocks in the forms titles. Same text renders correctly in labels.

So, I think I want to use the default font, and let Windows handle it. I think I've read that everything from XP on will handle it. Windows 2000 won't, which is a shame, but what ever, I don't know what font I should use, and whatever font it is was excluded from VS. I don't know how to add it without getting lots of error messages about protected memory.

Now the problem is, resizing the labels in the form, since the translated text is often larger.

Also, I don't have a support team to do this for me, but I could maybe install extra copies of Windows 7, and change to the Japanese language pack and try to run it. But that becomes a major pain. I thought I read that you could change the language while the application is running, but that doesn't seem to be true. MS docs talk about how to do this, but Windows 7 also tells you that you have to log out to change the language.

MS Gothic seems to work, and it's part of VS 2008, but the title bar is still squares. That's really odd, since the same glyphs are in the winform. Is that because title bar fonts are set at a system level, and not the application level?

Sorry for being wordy. Not sure that there is even a question here. Just trying to get it all out, so maybe this will help someone else down the line.

Answer 1

Any .NET code, including Windows Forms, uses Unicode encoded as UTF16. Your problem isn't likely to be an encoding issue, that produces question marks instead of squares. Getting a square indicates you are using a font that is missing the required glyph to display the Japanese character.

You can use the charmap.exe applet to find out what glyphs are supported by the font you use. If they are missing then the operating system is forced to fall back to a substitute font and fail at it when it cannot find one. Displaying squares is then all it can do. An old operating system version is a very likely cause for this mishap, particularly so for XP without the optional East Asian fonts installed.

Do note that this font problem is very unlikely to be an issue on a machine that boots the Japanese version of Windows. It will have to proper fonts to display Japanese text of course. You can get specific language versions of Windows through an MSDN subscription. At least get one for your QA staff so they can verify that everything works correctly.

Answer 2

Here's a great read on your issues from Joel Spolsky.