windows cmd pipe not unicode even with /U switch

Question

I have a little c# console program that outputs some text using Console.WriteLine. I then pipe this output into a textfile like:

c:myprogram > textfile.txt

However, the file is always an ansi text file, even when I start cmd with the /u switch. cmd /? says about the /u switch:

/U Causes the output of internal commands to a pipe or file to be Unicode

And it indeed makes a difference, when I do an

c:echo "foo" > text.txt

the text.txt is unicode (without BOM)

I wonder why piping the output of my console program into a new file does not create an unicode file likewise and how i could change that?

I just use Windows Power Shell (which produces a unicode file with correct BOM), but I'd still like to know how to do it with cmd.

Thanks!

Answer 1

The /U switch, as the documentation says, affects whether internal commands generate Unicode output. Your program is not one of cmd.exe's internal commands, so the /U option does not affect it.

To create a Unicode text file, you need to make sure your program is generating Unicode text.

Even that may not be enough, though. I came across this blog from Junfeng Zhang describing how to write Unicode text in a console program. It checks the file type of the standard output handle. For character files (a console or LPT port), it calls WriteFileW. For all other types of handles (including disk files and pipes), it converts the output string to the console's current code page. I'm afraid I don't know how that translates into .Net terms, though.

Answer 2

I had a look how mscorlib implements Console.WriteLine, and it seems to decide on which text output encoding to use based on a call to GetConsoleOutPutCP. So I'm guessing (but have not yet confimed) that the codepage returned is a differnt one for a PS console than for a cmd console so that my program indeed only outputs ansi when running from cmd.