No console output when using AllocConsole and target architecture x86

Question

I have a WinForms project, and if the user want's a debug console, I allocate a console with AllocConsole().

All console output works normally with the target architecture set to "Any CPU", but when I change it to "x86" it doesn't output anything (Console.Read() still works as expected). If I open the EXE directly, the output works. It looks like Visual Studio redirects it into it's own "Output" window.

I also tried this answer, but it didn't work, I also tried Console.SetOut(GetStdHandle(-11)), which didn't work either.

Setting the target architecture to 'Any CPU' is no option for me.

So here are my two questions:

• Why is this only the case when the target architecture is set to x86?
• How can I output to my console when running inside of Visual Studio?

Answer 1

When "Enable native code debugging" is enabled, output from consoles crated with AllocConsole is redirected to the debug output window instead.

The reason this only happens in x86 and not AnyCPU is because you can only debug native code in an x86 application.

Note that this behavior only occurs with consoles created with AllocConsole. A console application's output is not redirected.

EDIT: The other reason for the console not outputting text is when you've written to the console before calling AllocConsole.

Regardless of the reason, this code will restore output if it was redirected, and reopen the console in case it's invalid. It uses the magic number 7 which is what the handle of stdout usually equals to.

using System;
using System.IO;
using System.Runtime.InteropServices;

public static class ConsoleHelper
{
    public static void CreateConsole()
    {
        AllocConsole();

        // stdout's handle seems to always be equal to 7
        IntPtr defaultStdout = new IntPtr(7);
        IntPtr currentStdout = GetStdHandle(StdOutputHandle);

        if (currentStdout != defaultStdout)
            // reset stdout
            SetStdHandle(StdOutputHandle, defaultStdout);

        // reopen stdout
        TextWriter writer = new StreamWriter(Console.OpenStandardOutput()) 
        { AutoFlush = true };
        Console.SetOut(writer);
    }

    // P/Invoke required:
    private const UInt32 StdOutputHandle = 0xFFFFFFF5;
    [DllImport("kernel32.dll")]
    private static extern IntPtr GetStdHandle(UInt32 nStdHandle);
    [DllImport("kernel32.dll")]
    private static extern void SetStdHandle(UInt32 nStdHandle, IntPtr handle);
    [DllImport("kernel32")]
    static extern bool AllocConsole();
}

See How to detect if Console.In (stdin) has been redirected? for another way to detect if the console handles have been redirected.

Answer 2

None of the earlier answers worked well for me with VS2017 and Windows 10 (for instance they failed if launch app in debug mode).

Below you can find a little bit enhanced code. Idea is the same, but magic numbers are removed (Ceztko already mentioned that) and all necessary in\out streams are initialized.

This code works for me if create a new console (alwaysCreateNewConsole = true).

Attaching to console of parent process (alwaysCreateNewConsole = false) has several drawbacks. For example I was unable to completely mimic behavior of console app launched from cmd. And I'm not sure that it is possible at all.

And most important: after revision of Console class I reconsidered general idea of using Console class with manually created console. It works well (I hope) for most of the cases, but can bring a lot of pain in future.

    static class WinConsole
    {
        static public void Initialize(bool alwaysCreateNewConsole = true)
        {
            bool consoleAttached = true;
            if (alwaysCreateNewConsole
                || (AttachConsole(ATTACH_PARRENT) == 0
                && Marshal.GetLastWin32Error() != ERROR_ACCESS_DENIED))
            {
                consoleAttached = AllocConsole() != 0;
            }

            if (consoleAttached)
            {
                InitializeOutStream();
                InitializeInStream();
            }
        }

        private static void InitializeOutStream()
        {
            var fs = CreateFileStream("CONOUT$", GENERIC_WRITE, FILE_SHARE_WRITE, FileAccess.Write);
            if (fs != null)
            {
                var writer = new StreamWriter(fs) { AutoFlush = true };
                Console.SetOut(writer);
                Console.SetError(writer);
            }
        }

        private static void InitializeInStream()
        {
            var fs = CreateFileStream("CONIN$", GENERIC_READ, FILE_SHARE_READ, FileAccess.Read);
            if (fs != null)
            {
                Console.SetIn(new StreamReader(fs));
            }
        }

        private static FileStream CreateFileStream(string name, uint win32DesiredAccess, uint win32ShareMode,
                                FileAccess dotNetFileAccess)
        {
            var file = new SafeFileHandle(CreateFileW(name, win32DesiredAccess, win32ShareMode, IntPtr.Zero, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, IntPtr.Zero), true);
            if (!file.IsInvalid)
            {
                var fs = new FileStream(file, dotNetFileAccess);
                return fs;
            }
            return null;
        }

        #region Win API Functions and Constants
        [DllImport("kernel32.dll",
            EntryPoint = "AllocConsole",
            SetLastError = true,
            CharSet = CharSet.Auto,
            CallingConvention = CallingConvention.StdCall)]
        private static extern int AllocConsole();

        [DllImport("kernel32.dll",
            EntryPoint = "AttachConsole",
            SetLastError = true,
            CharSet = CharSet.Auto,
            CallingConvention = CallingConvention.StdCall)]
        private static extern UInt32 AttachConsole(UInt32 dwProcessId);

        [DllImport("kernel32.dll",
            EntryPoint = "CreateFileW",
            SetLastError = true,
            CharSet = CharSet.Auto,
            CallingConvention = CallingConvention.StdCall)]
        private static extern IntPtr CreateFileW(
              string lpFileName,
              UInt32 dwDesiredAccess,
              UInt32 dwShareMode,
              IntPtr lpSecurityAttributes,
              UInt32 dwCreationDisposition,
              UInt32 dwFlagsAndAttributes,
              IntPtr hTemplateFile
            );

        private const UInt32 GENERIC_WRITE = 0x40000000;
        private const UInt32 GENERIC_READ = 0x80000000;
        private const UInt32 FILE_SHARE_READ = 0x00000001;
        private const UInt32 FILE_SHARE_WRITE = 0x00000002;
        private const UInt32 OPEN_EXISTING = 0x00000003;
        private const UInt32 FILE_ATTRIBUTE_NORMAL = 0x80;
        private const UInt32 ERROR_ACCESS_DENIED = 5;

        private const UInt32 ATTACH_PARRENT = 0xFFFFFFFF;

        #endregion
    }

Answer 3

Following worked for me in vs 2015, none worked from other answers:

Source: https://social.msdn.microsoft.com/profile/dmitri567/?ws=usercard-mini

using System;   
using System.Windows.Forms;   
using System.Text;   
using System.IO;   
using System.Runtime.InteropServices;   
using Microsoft.Win32.SafeHandles;   

namespace WindowsApplication   
{   
    static class Program   
    {   
        [DllImport("kernel32.dll",   
            EntryPoint = "GetStdHandle",   
            SetLastError = true,   
            CharSet = CharSet.Auto,   
            CallingConvention = CallingConvention.StdCall)]   
        private static extern IntPtr GetStdHandle(int nStdHandle);   
        [DllImport("kernel32.dll",   
            EntryPoint = "AllocConsole",   
            SetLastError = true,   
            CharSet = CharSet.Auto,   
            CallingConvention = CallingConvention.StdCall)]   
        private static extern int AllocConsole();   
        private const int STD_OUTPUT_HANDLE = -11;   
        private const int MY_CODE_PAGE = 437;   

        static void Main(string[] args)   
        {   
            Console.WriteLine("This text you can see in debug output window.");   

            AllocConsole();   
            IntPtr stdHandle=GetStdHandle(STD_OUTPUT_HANDLE);   
            SafeFileHandle safeFileHandle = new SafeFileHandle(stdHandle, true);   
            FileStream fileStream = new FileStream(safeFileHandle, FileAccess.Write);   
            Encoding encoding = System.Text.Encoding.GetEncoding(MY_CODE_PAGE);   
            StreamWriter standardOutput = new StreamWriter(fileStream, encoding);   
            standardOutput.AutoFlush = true;   
            Console.SetOut(standardOutput);   

            Console.WriteLine("This text you can see in console window.");   

            MessageBox.Show("Now I'm happy!");   
        }   
    }   
}  

Answer 4

I also had this problem. Every time I tried to debug my app, the console was blank. Strangely, launching the exe without the debugger worked fine.

I found that I had to Enable the Visual Studio hosting process from the project's Debug menu.

Stephen is correct that Enable native code debugging does redirect the console to the Output window. However, regardless of the native code debugging setting, I saw absolutely no output in either place until I enabled the Visual Studio hosting process.

This could have been the reason that merely disabling native code debugging did not solve your issue.

Answer 5

Just wanted to post the answer from Visual studio Developer community. https://developercommunity.visualstudio.com/content/problem/12166/console-output-is-gone-in-vs2017-works-fine-when-d.html Go to this link and look at the answer from Ramkumar Ramesh. I have tested this code in VS 2017. I spent one day to find this answer. Hope it helps you as well.

Edit-- As suggessted by Mike to include some description. I would like to suggesst some corrections in Zuniar answer. He tested with VS 2015. But that would not work in VS 2017. Instead of GetStdHandle, Please use CreateFile reference from kernel32.dll

IntPtr stdHandle = CreateFile("CONOUT$", GENERIC_WRITE, FILE_SHARE_WRITE, 0, 
OPEN_EXISTING, 0, 0);

Before adding above code, please declare

[DllImport("kernel32.dll", SetLastError = true)]
private static extern IntPtr CreateFile(string lpFileName, uint 
dwDesiredAccess, uint dwShareMode, uint lpSecurityAttributes, uint 
dwCreationDisposition, uint dwFlagsAndAttributes, uint hTemplateFile);

private const int MY_CODE_PAGE = 437;
private const uint GENERIC_WRITE = 0x40000000;
private const uint FILE_SHARE_WRITE = 0x2;        
private const uint OPEN_EXISTING = 0x3;

i have taken this code from the given link.