How to make non-interactive graphical overlay on top of another program in c#?

Question

To give some background, I am developing a piece of software that assists players with the game Star Wars: The old republic. The game has very limited user interface capabilities, so i am developing an external application that will parse the log in real time, and output visual clues to help the user maximize their in-game performance. For example, if a character gets a certain "buff" the combat-log will show it, and i want to place a visual clue on the screen (so the user doesn't need to pay attention to small icons at the perimeter of the screen).

Before i begin, i wanted to create a few "proof of concept" scripts for myself to figure out how i am going to hand the major parts. The one i am stuck on is where i have my question:

I need to be able to show a graphic, likely a PNG file with transparency, on the screen over the game. The user needs to be able to click through that image so they can continue to interact with the game. I am a bit lost on how to go about that. The requirements would be:

• Show an image (or multiple images, for that matter)
• Have that image stick over-top the other application, even without application "focus"
• Have that image be non-interactive (or click-through-able).
• I am developing the application in C#

Any guidance on where to begin would be very much appreciated!

Answer 1

I have started looking at doing some similar things, so this might be able to give you a start.

For a first version, you might start by looking at the "AllowTransparency", and "TransparencyKey" , and "TopMost" properties of Form.

(I have found that the TransparencyKey doesn't work with White (255,255,255), but that specific non-white colors work fine...not sure why).

This would work as a click-throughable form that would stand above other forms...but since it is transparent, you can't display images in the transparent part. But if all you need is a hud that fits around the target application, this might be the easiest way.

If this top level form doesn't end up in front of the game...you might try putting the game in Windowed mode.

When running in full-screen mode, games generally draw to the screen directly through ActiveX, Direct3D, OpenGL, DirectDraw, etc.

Drawing on top of these would require injecting code into the DirectX, OpenGL, orother engine's draw/update/refresh function (basically tell DirectX3D to draw your stuff at the end of each draw cycle). There are some existing software that does this: for example, Steam Overlay, fraps, xfire.

A quick google search found "Game Overlay" which although I haven't downloaded or tried, says that it can overlay form applications on top of games for you.

(Seems that that program is under a company that was just dissolved, and I couldn't seem to get it to work for me anyway...)

It is possible to create a form that is not completely transparent but is click throughable by making native Windows calls..I'll see if I can create an example over the next few days.

---

I found an old test project and cleaned it up a bit.

Basically when run it will draw 500 random red lines to the front of the screen that are clickthrough-able. Then it draws 1000 random white lines (i.e. erases). Then repeats.

In writing the code I wanted to get a proof of concept for a couple of things: How to be able to draw on the full surface of a form, How to programmatically make the form become full size over multiple screens, How to make use of Background Workers, and How this proof of concept might work as a transparent overlay.

Instructions:

• Create a new Windows Forms Project named TranparentOverlay_simpleExample
• In design view, set the following properties on Form1:
  • BackColor: White
  • FormBorderStyle: None
  • Location: -1280, 0 (i.e. the top left corner of your screens, with one screen probably just 0,0)
  • TopMost: True
  • TransparencyKey: White
  • WindowState: Maximized

Now enter the code view for Form1 and replace it with the following:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Threading;
using System.Runtime.InteropServices;
namespace TransparentOverlay_simpleExample
{

    public partial class Form1 : Form
    {
        BackgroundWorker bw = new BackgroundWorker();
        Random rand = new Random(DateTime.Now.Millisecond);

        [DllImport("user32.dll")]
        [return: MarshalAs(UnmanagedType.Bool)]
        static extern bool BringWindowToTop(IntPtr hWnd);

        [DllImport("user32.dll")]
        [return: MarshalAs(UnmanagedType.Bool)]
        static extern bool SetForegroundWindow(IntPtr hWnd);

        [DllImport("user32.dll", EntryPoint = "SetWindowPos")]
        public static extern IntPtr SetWindowPos(IntPtr hWnd, int hWndInsertAfter, int x, int Y, int cx, int cy, int wFlags);



        [DllImport("user32.dll")]
        static extern int SendMessage(IntPtr hWnd, uint wMsg, UIntPtr wParam, IntPtr lParam); //used for maximizing the screen

        const int WM_SYSCOMMAND = 0x0112; //used for maximizing the screen.
        const int myWParam = 0xf120; //used for maximizing the screen.
        const int myLparam = 0x5073d; //used for maximizing the screen.


        int oldWindowLong;

        [Flags]
        enum WindowStyles : uint
        {
            WS_OVERLAPPED = 0x00000000,
            WS_POPUP = 0x80000000,
            WS_CHILD = 0x40000000,
            WS_MINIMIZE = 0x20000000,
            WS_VISIBLE = 0x10000000,
            WS_DISABLED = 0x08000000,
            WS_CLIPSIBLINGS = 0x04000000,
            WS_CLIPCHILDREN = 0x02000000,
            WS_MAXIMIZE = 0x01000000,
            WS_BORDER = 0x00800000,
            WS_DLGFRAME = 0x00400000,
            WS_VSCROLL = 0x00200000,
            WS_HSCROLL = 0x00100000,
            WS_SYSMENU = 0x00080000,
            WS_THICKFRAME = 0x00040000,
            WS_GROUP = 0x00020000,
            WS_TABSTOP = 0x00010000,

            WS_MINIMIZEBOX = 0x00020000,
            WS_MAXIMIZEBOX = 0x00010000,

            WS_CAPTION = WS_BORDER | WS_DLGFRAME,
            WS_TILED = WS_OVERLAPPED,
            WS_ICONIC = WS_MINIMIZE,
            WS_SIZEBOX = WS_THICKFRAME,
            WS_TILEDWINDOW = WS_OVERLAPPEDWINDOW,

            WS_OVERLAPPEDWINDOW = WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU | WS_THICKFRAME | WS_MINIMIZEBOX | WS_MAXIMIZEBOX,
            WS_POPUPWINDOW = WS_POPUP | WS_BORDER | WS_SYSMENU,
            WS_CHILDWINDOW = WS_CHILD,

            //Extended Window Styles

            WS_EX_DLGMODALFRAME = 0x00000001,
            WS_EX_NOPARENTNOTIFY = 0x00000004,
            WS_EX_TOPMOST = 0x00000008,
            WS_EX_ACCEPTFILES = 0x00000010,
            WS_EX_TRANSPARENT = 0x00000020,

            //#if(WINVER >= 0x0400)

            WS_EX_MDICHILD = 0x00000040,
            WS_EX_TOOLWINDOW = 0x00000080,
            WS_EX_WINDOWEDGE = 0x00000100,
            WS_EX_CLIENTEDGE = 0x00000200,
            WS_EX_CONTEXTHELP = 0x00000400,

            WS_EX_RIGHT = 0x00001000,
            WS_EX_LEFT = 0x00000000,
            WS_EX_RTLREADING = 0x00002000,
            WS_EX_LTRREADING = 0x00000000,
            WS_EX_LEFTSCROLLBAR = 0x00004000,
            WS_EX_RIGHTSCROLLBAR = 0x00000000,

            WS_EX_CONTROLPARENT = 0x00010000,
            WS_EX_STATICEDGE = 0x00020000,
            WS_EX_APPWINDOW = 0x00040000,

            WS_EX_OVERLAPPEDWINDOW = (WS_EX_WINDOWEDGE | WS_EX_CLIENTEDGE),
            WS_EX_PALETTEWINDOW = (WS_EX_WINDOWEDGE | WS_EX_TOOLWINDOW | WS_EX_TOPMOST),
            //#endif /* WINVER >= 0x0400 */

            //#if(WIN32WINNT >= 0x0500)

            WS_EX_LAYERED = 0x00080000,
            //#endif /* WIN32WINNT >= 0x0500 */

            //#if(WINVER >= 0x0500)

            WS_EX_NOINHERITLAYOUT = 0x00100000, // Disable inheritence of mirroring by children
            WS_EX_LAYOUTRTL = 0x00400000, // Right to left mirroring
            //#endif /* WINVER >= 0x0500 */

            //#if(WIN32WINNT >= 0x0500)

            WS_EX_COMPOSITED = 0x02000000,
            WS_EX_NOACTIVATE = 0x08000000
            //#endif /* WIN32WINNT >= 0x0500 */

        }

        public enum GetWindowLongConst
        {
            GWL_WNDPROC = (-4),
            GWL_HINSTANCE = (-6),
            GWL_HWNDPARENT = (-8),
            GWL_STYLE = (-16),
            GWL_EXSTYLE = (-20),
            GWL_USERDATA = (-21),
            GWL_ID = (-12)
        }

        public enum LWA
        {
            ColorKey = 0x1,
            Alpha = 0x2,
        }

        [DllImport("user32.dll", SetLastError = true)]
        static extern int GetWindowLong(IntPtr hWnd, int nIndex);

        [DllImport("user32.dll")]
        static extern int SetWindowLong(IntPtr hWnd, int nIndex, int dwNewLong);

        [DllImport("user32.dll")]
        static extern bool SetLayeredWindowAttributes(IntPtr hwnd, uint crKey, byte bAlpha, uint dwFlags);

        /// <summary>
        /// Make the form (specified by its handle) a window that supports transparency.
        /// </summary>
        /// <param name="Handle">The window to make transparency supporting</param>
        public void SetFormTransparent(IntPtr Handle)
        {
            oldWindowLong = GetWindowLong(Handle, (int)GetWindowLongConst.GWL_EXSTYLE);
            SetWindowLong(Handle, (int)GetWindowLongConst.GWL_EXSTYLE, Convert.ToInt32( oldWindowLong | (uint)WindowStyles.WS_EX_LAYERED | (uint)WindowStyles.WS_EX_TRANSPARENT));
        }

        /// <summary>
        /// Make the form (specified by its handle) a normal type of window (doesn't support transparency).
        /// </summary>
        /// <param name="Handle">The Window to make normal</param>
        public void SetFormNormal(IntPtr Handle)
        {
            SetWindowLong(Handle, (int)GetWindowLongConst.GWL_EXSTYLE, Convert.ToInt32(oldWindowLong | (uint)WindowStyles.WS_EX_LAYERED));
        }

        /// <summary>
        /// Makes the form change White to Transparent and clickthrough-able
        /// Can be modified to make the form translucent (with different opacities) and change the Transparency Color.
        /// </summary>
        public void SetTheLayeredWindowAttribute()
        {
            uint transparentColor = 0xffffffff;

            SetLayeredWindowAttributes(this.Handle, transparentColor, 125, 0x2);

            this.TransparencyKey = Color.White;
        }

        /// <summary>
        /// Finds the Size of all computer screens combined (assumes screens are left to right, not above and below).
        /// </summary>
        /// <returns>The width and height of all screens combined</returns>
        public static Size getFullScreensSize()
        {
            int height = int.MinValue;
            int width = 0;

            foreach (Screen screen in System.Windows.Forms.Screen.AllScreens)
            {
                //take largest height
                height = Math.Max(screen.WorkingArea.Height, height);

                width += screen.Bounds.Width;
            }

            return new Size(width, height);
        }

        /// <summary>
        /// Finds the top left pixel position (with multiple screens this is often not 0,0)
        /// </summary>
        /// <returns>Position of top left pixel</returns>
        public static Point getTopLeft()
        {
            int minX = int.MaxValue;
            int minY = int.MaxValue;

            foreach (Screen screen in System.Windows.Forms.Screen.AllScreens)
            {
                minX = Math.Min(screen.WorkingArea.Left, minX);
                minY = Math.Min(screen.WorkingArea.Top, minY);
            }

            return new Point( minX, minY );
        }

        public Form1()
        {
            InitializeComponent();

            MaximizeEverything();

            SetFormTransparent(this.Handle);

            SetTheLayeredWindowAttribute();

            BackgroundWorker tmpBw = new BackgroundWorker();
            tmpBw.DoWork += new DoWorkEventHandler(bw_DoWork);

            this.bw = tmpBw;

            this.bw.RunWorkerAsync();
        }

        private void MaximizeEverything()
        {
            this.Location = getTopLeft();
            this.Size = getFullScreensSize();

            SendMessage(this.Handle, WM_SYSCOMMAND, (UIntPtr)myWParam, (IntPtr)myLparam);
        }

        private void bw_DoWork(object sender, DoWorkEventArgs e)
        {
            BackgroundWorker worker = sender as BackgroundWorker;

            int numRedLines = 500;
            int numWhiteLines = 1000;

            Size fullSize = getFullScreensSize();
            Point topLeft = getTopLeft();

            using (Pen redPen = new Pen(Color.Red, 10f), whitePen = new Pen(Color.White, 10f)) {
                using (Graphics formGraphics = this.CreateGraphics()) {

                    while (true) {

                        bool makeRedLines = true;

                        for (int i = 0; i < numRedLines + numWhiteLines; i++)
                        {

                            if (i > numRedLines)
                            {
                                makeRedLines = false;
                            }

                            //Choose points for random lines...but don't draw over the top 100 px of the screen so you can 
                            //still find the stop run button.
                            int pX = rand.Next(0, (-1 * topLeft.X) + fullSize.Width);
                            int pY = rand.Next(100, (-1 * topLeft.Y) + fullSize.Height);

                            int qX = rand.Next(0, (-1 * topLeft.X) + fullSize.Width);
                            int qY = rand.Next(100, (-1 * topLeft.Y) + fullSize.Height);

                            if (makeRedLines)
                            {
                                formGraphics.DrawLine(redPen, pX, pY, qX, qY);
                            }
                            else
                            {
                                formGraphics.DrawLine(whitePen, pX, pY, qX, qY);
                            }

                            Thread.Sleep(10);
                        }
                    }
                }
            }
        }
    }
}

The lists of Enums are values used in native windows calls and converting RGB colors like White into uints makes dealing with native Windows a bit of a pain.

But, at last, we now have an invisible canvas that covers all screens, and we can draw to it just as with any other graphics object (so it is just about as easy to draw text or pictures as lines).

(I think that if you draw a translucent picture to the graphics object, that you could make yourself a translucent overlay rather than fully opaque/transparent overlays).

This example can't place overlays over fullscreen 3d games, but works fine for those same games run in Windowed mode.

(P.S. I just tested this in Team Fortress 2, it draws over it in Windowed mode, but not fullscreen, so I'm guessing The Old Republic will be similar).

---

The following links might be useful to anyone trying to hook into the drawing routine for Direct3D versions 9, 10, and 11.

http://spazzarama.com/2011/03/14/c-screen-capture-and-overlays-for-direct3d-9-10-and-11-using-api-hooks/

https://github.com/spazzarama/Direct3DHook

It doesn't provide a full featured overlay, but the example project above successfully writes the frames per second on top of Team Fortress 2 for me. It has good instructions on how to start using it. It should guide you through the process of setting up SlimDX Runtime and EasyHook.

Answer 2

The way that I have done this in the past is to get the handle for the main window, often the only way to do it is to traverse the entire window list, looking for the one with the title that you want. That is problematic if there are two instances of the game open at the same time with the same title.

Once you have that window handle, you can add visual elements over the top of what is already there by specifying how many pixels over, how many pixels down and how many layers "out" (the z-index) you want it to be relative to the upper left pixel of the window.

One approach to the multiple window is to start your c# program that looks to see if there is an instance of the game already being played, issuing a message and terminating if there is, if not, then starting an instance of the game your self as a child process. I think you can get back the hwin at that time, but if not you can search the window list for the target title after the game is running.