gUnicorn/gevent: requests library is throwing RecursionError: maximum recursion depth exceeded

Question

I have a Flask app that deployed using gunicorn and gevent. I'm have put

import gevent.monkey
gevent.monkey.patch_all()

at the top of app.py file.

But I'm still getting the follow warning and errors:

MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['requests.packages.urllib3.util


  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/api.py", line 110, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 595, in urlopen
    chunked=chunked)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 352, in _make_request
    self._validate_conn(conn)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 831, in _validate_conn
    conn.connect()
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 289, in connect
    ssl_version=resolved_ssl_version)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/packages/urllib3/util/ssl_.py", line 291, in ssl_wrap_socket
    ciphers=ciphers)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/requests/packages/urllib3/util/ssl_.py", line 254, in create_urllib3_context
    context.options |= options
  File "/opt/python/3.6.3/lib/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/opt/python/3.6.3/lib/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/opt/python/3.6.3/lib/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 306 more times]
RecursionError: maximum recursion depth exceeded

Answer 1

We encountered the similar issue and found out that we were wrapping gunicorn with ddtrace-run(datadog tracer) and ddtrace-run was calling urllib3 even before gevent can monkeypatch it. More details about that error here dd-trace-py/issues/506. To solve the bug we upgraded to latest version of ddtrace-run.

ddtrace-run gunicorn -b 0.0.0.0:${PORT} --log-level=${GUNICORN_LOG_LEVEL} ... 'app.main:main()'

If you have added gevent.monkey.patch_all() at the top of app.py then check carefully how you are invoking the gunicorn entry-point. There will be something that is coming prior to gevent and invoking SSL.

Answer 2

You could solve it with a gunicorn config file, and start gunicorn with this file. And add the below lines in the config file.

import gevent.monkey
gevent.monkey.patch_all()

For more detail you can refer to this answer