What's the correct way to use a unix domain socket in requests framework?

Question

Usually, doing a post request using requests framework is done by:

payload = {'key1': 'value1', 'key2': 'value2'}
r = requests.post("http://httpbin.org/post", data=payload)

But: How do I connect to a unix socket instead of doing a TCP connection?

On a related note, how to encode domain path in the URL?

• libcurl allows application to supply own socket on which to perform request
• LDAP invented own scheme ldapi where socket name is %-encoded in host field
• httpie uses http+unix scheme and %-encoded path in host field

These are some examples, but is there an RFC or established best practice?

Answer 1

There's no need to reinvent the wheel:

https://github.com/msabramo/requests-unixsocket

URL scheme is http+unix and socket path is percent-encoded into the host field:

import requests_unixsocket

session = requests_unixsocket.Session()

# Access /path/to/page from /tmp/profilesvc.sock
r = session.get('http+unix://%2Ftmp%2Fprofilesvc.sock/path/to/page')
assert r.status_code == 200

Answer 2

If you are looking for a minimalistic and clean approach to this in Python 3, here's a working example that will talk to Ubuntu's snapd on a unix domain socket.

import requests
import socket
import pprint

from urllib3.connection import HTTPConnection
from urllib3.connectionpool import HTTPConnectionPool
from requests.adapters import HTTPAdapter


class SnapdConnection(HTTPConnection):
    def __init__(self):
        super().__init__("localhost")

    def connect(self):
        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        self.sock.connect("/run/snapd.socket")


class SnapdConnectionPool(HTTPConnectionPool):
    def __init__(self):
        super().__init__("localhost")

    def _new_conn(self):
        return SnapdConnection()


class SnapdAdapter(HTTPAdapter):
    def get_connection(self, url, proxies=None):
        return SnapdConnectionPool()


session = requests.Session()
session.mount("http://snapd/", SnapdAdapter())
response = session.get("http://snapd/v2/system-info")
pprint.pprint(response.json())

Answer 3

You can use socat to create a TCP to UNIX socket proxy, something like:

socat TCP-LISTEN:80,reuseaddr,fork UNIX-CLIENT:/tmp/foo.sock

And then send your http requests to that proxy. The server listening on UNIX socket /tmp/foo.sock still has to understand HTTP because socat does not do any message conversion.