Preserve non-ascii characters between std::string and QString

Question

In my program the user can either provide a filename on the command line or using a QFileDialog. In the first case, I have a char* without any encoding information, in the second I have a QString.

To store the filename for later use (Recent Files), I need it as a QString. But to open the file with std::ifstream, I need a std::string.

Now the fun starts. I can do:

filename = QString::fromLocal8Bit(argv[1]);

later on, I can do:

std::string fn = filename.toLocal8Bit().constData();

This works for most characters, but not all. For example, the word Раи́са will look the same after going through this conversion, but, in fact, have different characters. So while I can have a Раи́са.txt, and it will display Раи́са.txt, it will not find the file in the filesystem. Most letters work, but и́ doesnt. (Note that it does work correctly when the file was chosen in the QFileDialog. It does not when it originated from the command line.)

Is there any better way to preserve the filename? Right now I obtain it in whatever native encoding, and can pass-on in the same encoding, without knowing it. At least so I thought.

Answer 1

'и́' is not an ASCII character, that is to say it has no 8-bit representation. How it is represented in argv[1] then is OS dependent. But it's not getting represented in just one char.

The fromLocal8bit uses the same QTextCodec::codecForLocale as toLocal8bit. And as you say your std::string will hold "Раи́са.txt" so that's not the problem.

Depending on how your OS defined std::ifstream though std::ifstream may expect each char to be it's own char and not go through the OS's translation. I expect that you are on Windows since you are seeing this problem. In which case you should use the std::wstring implementation of std::fstream which is Microsoft specific: http://msdn.microsoft.com/en-us/library/4dx08bh4.aspx

You can get a std::wstring from QString by using: toStdWString

See here for more info: fstream::open() Unicode or Non-Ascii characters don't work (with std::ios::out) on Windows

EDIT:

A good cross-platform option for projects with access to it is Boost::Filesystem. ypnos Mentions File-Streams as specifically pertinent.