How can I get unicode characters from robocopy process standard ouput in c#

Question

Our application runs various actions and displays the output in a log window. One action uses robocopy to copy files between folders.

This works ok until the robocopy output contains unicode characters. I understand that I need to use the /unicode option but all I seem to get back is gibberish.

Here's my simplified code sample:

class Program
{
    static void Main(string[] args)
    {


        StreamReader outputReader = null;
        StreamReader errorReader = null;


        using (Process process = new Process())
        {

            Encoding encoding = Encoding.Default;

            if (encoding != null)
            {
                process.StartInfo.StandardOutputEncoding = encoding;
                process.StartInfo.StandardErrorEncoding = encoding;
            }

            process.StartInfo.FileName = @"C:\Windows\system32\robocopy.exe";
            process.StartInfo.Arguments = @"""D:\temp\некоторые случайные папки"" ""D:\temp\другой случайные папки"" /unicode";
            process.StartInfo.ErrorDialog = false;
            process.StartInfo.LoadUserProfile = false;
            process.StartInfo.RedirectStandardError = true;
            process.StartInfo.RedirectStandardOutput = true;
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.CreateNoWindow = true;

            process.StartInfo.WorkingDirectory = @"D:\temp\некоторые случайные папки";



            bool processStarted = process.Start();
            if (processStarted)
            {
                //Get the output stream
                outputReader = process.StandardOutput;
                errorReader = process.StandardError;
                process.WaitForExit();

                string standardOutput = outputReader.ReadToEnd();
                string errorOutput = errorReader.ReadToEnd();
                if (!string.IsNullOrEmpty(standardOutput))
                {

                    byte[] bytes = encoding.GetBytes(standardOutput);
                    byte[] convertedBytes = Encoding.Convert(encoding, Encoding.UTF8, bytes);

                    string convertedStandardOutput = Encoding.UTF8.GetString(convertedBytes);

                    Console.Write("Standard output: ");
                    Console.WriteLine(convertedStandardOutput);
                }
                if (!string.IsNullOrEmpty(errorOutput))
                {
                    Console.Write("Error output: ");
                    Console.WriteLine(errorOutput);
                }
            }

        }

        Console.ReadKey();
    }
}

I've tried various encoding types and conversions to no avail. Here's the type of output I'm getting:

standardOutput: "ⴊⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭ †佒佂佃奐††㨠›††潒畢瑳䘠汩⁥潃祰映牯圠湩潤獷†††††††††††††††ⴊⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭਊ†瑓牡整⁤›潍摮祡‬㘱䴠牡档㈠㄰‵㐱ㄺ㨵㈵ †潓牵散㨠䐠尺整灭㽜㼿㼿㼿㼿㼠㼿㼿㼿㼿㼠㼿㼿ੜ††䐠獥⁴›㩄瑜浥屰㼿㼿㼿㼠㼿㼿㼿㼿㼠㼿㼿ੜ †䘠汩獥㨠⨠⨮ऊ†† 传瑰潩獮㨠⨠⸀⨀ ⼀唀一䤀䌀伀䐀䔀 ⼀䐀䌀伀倀夀㨀䐀䄀 ⼀䌀伀倀夀㨀䐀䄀吀 ⼀刀㨀㄀　　　　　　 ⼀圀㨀㌀　 ਀ⴊⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭਭऊ†††††††††〠䐉尺整灭㽜㼿㼿㼿㼿㼠㼿㼿㼿㼿㼠㼿㼿ੜⴊⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭⴭਭ †††††††潔慴†䌠灯敩⁤†歓灩数⁤䴠獩慭捴⁨†䘠䥁䕌⁄†䔠瑸慲ੳ††楄獲㨠††††ㄠ††††〠††††〠††††〠††††〠††††〠 †楆敬⁳›††††‰††††‰††††‰††††‰††††‰††††ਰ†䈠瑹獥㨠††††〠††††〠††††〠††††〠††††〠††††〠 †楔敭⁳›†㨰〰〺‰†㨰〰〺‰†††††††††††㨰〰〺‰†㨰〰〺ਰ†䔠摮摥㨠䴠湯慤ⱹㄠ‶慍捲⁨〲㔱ㄠ㨴㔱㔺ਲ"


convertedStandardOutput: "?????????????????????????????????????????†????††?›††??????????????†††††††††††††††?????????????????????????????????????????†????›??????????`?????†??????????????????????††??4›?????????????????†???????††????????? ???????? ????????? ????????? ????       ????  ??????????????????????????????????????????†††††††††????????????????????????????????????????????????????????????†††††††???†????†?????????†???/†????††???††††?††††?††††?††††?††††?††††??†???›††††‰††††‰††††‰††††‰††††‰††††?†????††††?††††?††††?††††?††††?††††??†???›†???‰†???‰†††††††††††???‰†????†????????????????????"

The output displayed when running in a command window is:

 ■
-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows
-------------------------------------------------------------------------------

  Started : Monday, 16 March 2015 14:24:01
   Source : D:\temp\некоторые случайные папки\
     Dest : D:\temp\другой случайные папки\

    Files : *.*

  Options : * . *   / U N I C O D E   / D C O P Y : D A   / C O P Y : D A T   / R : 1 0 0 0 0 0 0   / W : 3 0

------------------------------------------------------------------------------

                           0    D:\temp\некоторые случайные папки\

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         0         0         0         0
   Files :         0         0         0         0         0         0
   Bytes :         0         0         0         0         0         0
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00
   Ended : Monday, 16 March 2015 14:24:01

Any ideas?

Answer 1

It looks like the /UNICODE option is buggy: the only thing it affects in the console output is the Options : line. (You can tell that this part is Unicode from the spaces between the characters, which are caused by extra null bytes.) ROBOCOPY still seems to write everything else using the system code page. But the /UNICODE option does cause ROBOCOPY to write out a Unicode byte-order mark at the beginning of the output, so StreamReader switches to Unicode no matter what StandardOutputEncoding you set. The result: gibberish.

Instead of /UNICODE, use the /UNILOG option, which appears to work correctly (at least on Windows 8.1):

using (Process process = new Process())
{
    string logFileName = Path.GetTempFileName();
    process.StartInfo.FileName = @"C:\Windows\system32\robocopy.exe";
    process.StartInfo.Arguments = @"""D:\temp\некоторые случайные папки"" ""D:\temp\другой случайные папки"" /UNILOG:" + logFileName;
    process.StartInfo.ErrorDialog = false;
    process.StartInfo.LoadUserProfile = false;
    process.StartInfo.UseShellExecute = false;
    process.StartInfo.CreateNoWindow = true;
    process.StartInfo.WorkingDirectory = @"D:\temp\некоторые случайные папки";
    bool processStarted = process.Start();

    if (processStarted)
    {
        process.WaitForExit();
        string output = File.ReadAllText(logFileName);
        File.Delete(logFileName);
        // TODO: Do something with the output.
    }
}