Split a wstring by specified separator

Question

I have a std::wstring variable that contains a text and I need to split it by separator. How could I do this? I wouldn't use boost that generate some warnings. Thank you

EDIT 1 this is an example text:

hi how are you?

and this is the code:

typedef boost::tokenizer<boost::char_separator<wchar_t>, std::wstring::const_iterator, std::wstring> Tok;

boost::char_separator<wchar_t> sep;

Tok tok(this->m_inputText, sep);

for(Tok::iterator tok_iter = tok.begin(); tok_iter != tok.end(); ++tok_iter)
{
    cout << *tok_iter;
}

the results are:

1. hi
2. how
3. are
4. you
5. ?

I don't understand why the last character is always splitted in another token...

Answer 1

In your code, question mark appears on a separate line because that's how boost::tokenizer works by default.

If your desired output is four tokens ("hi", "how", "are", and "you?"), you could

a) change char_separator you're using to

boost::char_separator<wchar_t> sep(L" ", L"");

b) use boost::split which, I think, is the most direct answer to "split a wstring by specified character"

#include <string>
#include <iostream>
#include <vector>
#include <boost/algorithm/string.hpp>

int main()
{

        std::wstring m_inputText = L"hi how are you?";

        std::vector<std::wstring> tok;
        split(tok, m_inputText, boost::is_any_of(L" "));

        for(std::vector<std::wstring>::iterator tok_iter = tok.begin();
                        tok_iter != tok.end(); ++tok_iter)
        {
                std::wcout << *tok_iter << '\n';
        }

}

test run: https://ideone.com/jOeH9