Why does this solve the 'no $DISPLAY environment' issue with matplotlib?

Question

When running a code that uses the matplotlib library in my desktop PC, I have no issues using the line:

import matplotlib.pyplot as plt

far down the code, which is where I actually use the plotting functions.

If I run the code in a server though it will only work if I import matplotlib before, and force it to use the Agg backend. I.e., I have to add the following lines to the beginning of the code:

import matplotlib
# Force matplotlib to not use any Xwindows backend.
matplotlib.use('Agg')

(see this answer where this is explained). Otherwise the code will crash with TclError: no display name and no $DISPLAY environment variable (see this question for example).

The question is: why do I need to do this? The solution works perfectly, but I don't know why I don't have to do this in my desktop PC but I absolutely must when the code runs in the server.

Answer 1

X11 follows a client/server model, where the X server accepts requests for graphical output from client applications (e.g. interactive matplotlib sessions), and sends back user input from the keyboard, mouse etc. In order for this model to work, client applications need to know which X server to send their requests to. This is controlled by the $DISPLAY environment variable. In the case where you are connecting to a remote X session (for example over an SSH connection), the $DISPLAY variable in your remote session needs to point to your local X server.

The $DISPLAY variable is structured like this:

hostname:displaynumber.screennumber

Not all parts may be present - the hostname is usually omitted for local sessions, and the screen number is also omitted if there is only one screen. In a local terminal session on laptop, my $DISPLAY looks like this:

alistair@laptop:~$ echo $DISPLAY
:0

Provided that the remote server also supports X11, it's possible to open graphical windows on the remote machine and have them appear on your local machine using X11 forwarding. For an SSH connection you do this by passing the -X (or -Y) flag.

For example:

alistair@laptop:~$ ssh -X [email protected]
alistair@workstation:~$ echo $DISPLAY
localhost:10.0

The remote SSH server should take care of setting the $DISPLAY variable appropriately when you open the connection. In this particular case, localhost:10.0 is actually a 'proxy' X11 server running on the remote machine that listens on display 10 and relays commands to your local X server over the SSH connection (take a look at this if you're interested in the details).

Now you should be able to start a remote IPython session, import matplotlib using an interactive backend, and create plot windows which will then appear on your local machine. Since your keyboard/mouse input and the display output are now passing over an encrypted network connection, the plot windows will be less responsive than you're used to for a local session.

Another word of warning: if you have an IPython session open with an interactive matplotlib session running it is impossible to close the SSH connection without killing the IPython process. I also sometimes call matplotlib.use("Agg") before I start a long-running process that imports matplotlib - that way I can disconnect from the remote server without killing the process.