How to create Alt+Tab friendly fullscreen programs (like games) in Linux?

Question

I want to create an application where I draw on an window, either windowed or fullscreen, where I have the mouse grabbed but without intercepting any WM keyboard shortcuts, like Alt+Tab, and I also need to be notified whenever the user enter/leaves focus.

Common applications like Google Chrome, Firefox or gnome-terminal can handle this just fine (going fullscreen with F11, but still having Alt+Tab), but they do not grab the mouse.

SDL has a notorious poor handling of this use case: SDL_WM_GrabInput grabs the mouse but also intercepts WM shortcuts; and SDL_FULLSCREEN seems to have some sort of automatic grab by itself (don't ask me why).

A solution could be to write code for Alt+Tab myself, but this sucks (and doesn't help for other WM shortcuts, like changing to another workspace).

Another solution is to not call SDL_WM_GrabInput, but instead fake a grab: just hide the mouse pointer (with SDL_ShowCursor) and move it back to the center whenever the user moves. Which is ugly, but in practice works - except of course for SDL_FULLSCREEN, because it grabs automatically (unlike sane implementations). A fullscreen-capable SDL solution is this, but that's still not what I want. I don't want to have hacks to enable and disable grab, I want to grab the mouse but not grab the keyboard.

So I am mad with SDL and want to see alternatives. I would like to use SDL but this is not necessary.

This question seems to point out that what SDL actually does is to use XGrabKeyboard. By reading the man page it's not immediately clear to me whether you can grab the mouse without grabbing the keyboard (I never used Xlib myself).

I know how to make "fake fullscreen" with GTK (that is, of the Alt+Tab friendly, gnome-terminal variety). I imagine that doing this, coupled with mouse hiding and moving it back to the center ("fake grabbing") could do the trick, but this feels like too much duct tape. There must be a simpler way. (Also I don't want to add GTK as a dependency; but I'm also not sure if making raw Xlib calls is a good idea).

What's a good solution for this?

I need a Linux/X11 solution but it would be nice for it to be cross-platform - I know this can be solved smoothly on Windows, so perhaps there is a library that does exactly this. (also, I render with OpenGL but this is irrelevant)

PS: Perhaps I am having a bad understanding of this issue and I'm not asking the right question, so feel free to point out approaches I haven't considered.

Answer 1

I have been using Gtk and GtkGLExt for gaming on Linux these days (e.g., my Ludum Dare entries). Here (gtk.c Gist) is my code, I have released it under the FreeBSD license. You can see the code I've commented out for debugging window state changes. I should mention that this is part of a much larger framework which I use for things like Ludum Dare, and I prefer it to SDL precisely for the same reasons that you mention: the resulting applications conform more closely to users' expectations for native applications on their platforms (Linux, OS X, Windows). Needless to say, going this route is a lot of work.

Raw X11 programming, however, is a real mess. I estimate that it would take me a couple weeks of solid programming and reading docs to convert my Gtk code to X11 code; it's just not worth it for me but you may decide differently. (Many hours just to eliminate a dependency that everyone has installed anyway!) The Gtk dependency is not actually that outrageous, it's probably already loaded into memory anyway, and the ABI for 2.x is very stable so it won't hurt binary compatibility.

A big problem with X11 programming for games is that event handling is a total mess. You can't easily poll for events. The Xlib interface is blocking, so you have to use an arcane sequence of function calls to read data, check that there are events pending, and then only read events from the queue if they exist (otherwise your application will block until events appear). The xcb library is alternative to Xlib which is much better all around and supports a non-blocking interface, but it doesn't play well with OpenGL. So you could try to mix the two, or you could just use Gtk.

Let me give you a snippet of the Gtk code for fullscreen:

static void toggle_fullscreen()
{
    if (sg_gtk_status & SG_STATUS_VISIBLE) {
        if (sg_gtk_status & SG_STATUS_FULLSCREEN)
            gtk_window_unfullscreen(sg_window);
        else
            gtk_window_fullscreen(sg_window);
    }
}

Just imagine how much work this would be with the X11 interface: figure out the size of the screen (there might be more than one!), resize the window, change the ordering so that it's on top of everything else, change the decoration, and then respond intelligently when the user switches virtual desktops. Blah! (Actually, this describes how my code works on OS X, but the APIs are much nicer.)

More than that, if you're using X11, you'll have to learn how to react to the user in the expected way. X11 is from the 1980s. Gtk, by comparison, gives your application a swath of appropriate defaults from Gtk's UI designers. UI design is work. And remember: you can mix X11 with Gtk just fine.

The summary: X11 programming is for programmers with lots of spare time.

Notes: The GtkGLExt adds an annoying linker flag, -Wl,--export-dynamic. My build scripts remove the flag from pkg-config's output.

X11 experiences: I think I spent about a week trying to get the whole thing working in X11, and I ended up following lots of dead ends in the process. Learn from my failures.