Gunicorn - Where do Unix sockets come from?

Question

I am trying to configure Gunicorn to use Unix socket, but I can't find any documentation on how these sockets are created or how to troubleshoot them.

I am setting up a Django app, and I can successfully launch to http by specifying the port. However, when I try to use a Unix socket:

gunicorn myapp.wsgi:application --bind=unix:/appuser/myapp/gunicorn.sock

It returns with:

[2015-11-21 09:34:21 +0000] [24380] [INFO] Starting gunicorn 19.3.0
[2015-11-21 09:34:21 +0000] [24380] [ERROR] Retrying in 1 second.
[2015-11-21 09:34:22 +0000] [24380] [ERROR] Retrying in 1 second.
[2015-11-21 09:34:23 +0000] [24380] [ERROR] Retrying in 1 second.
[2015-11-21 09:34:24 +0000] [24380] [ERROR] Retrying in 1 second.
[2015-11-21 09:34:25 +0000] [24380] [ERROR] Retrying in 1 second.
[2015-11-21 09:34:26 +0000] [24380] [ERROR] Can't connect to /appuser/myapp/gunicorn.sock

The gunicorn.sock is located right where it is supposed to be, but the file is empty. I can't find any logs or anything. I am using Nginx on an AWS server and have set up all the relevant configurations according to the docs. I can post my configuration if it has anything to do with this.

How do I troubleshoot this?

Answer 1

You are using unix file system sockets. These are not regular files, so they might look empty when inspected with cat.

A peculiarity of unix file system sockets is that they must not exist before an application tries to bind for the path. There must neither be a directory, nor a file, nor a socket at that path before binding. You have to remove the socket; when binding, the socket will be created automatically. It must then, when the application terminates, be removed by the application (or a watchdog) before anyone else (including the re-started application) can bind to that path again.

Example:

>>> import socket
>>> s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM, 0)
>>> s.bind("foo.sock")
# now foo.sock exists
>>> s.close()
# foo.sock still exists
>>> s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM, 0)
>>> s.bind("foo.sock")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 98] Address already in use
# boom, the socket must be unlinked manually
>>> import os
>>> os.unlink("foo.sock")
>>> s.bind("foo.sock")
>>> s.close()

To avoid these issues, I would use nginx with a http-based reverse proxy to the application and bind the application to 127.0.0.1:someport.