Is it possible to run multiple gunicorn workers with aiohttp?

Question

I'm trying to run a gunicorn server with multiple workers, using aiohttp for asynchronous processing.

When running the service with one worker process everything works, but when using more than one process the service fails

It appears to be because of the bug: https://bugs.python.org/issue22087 Is there a work around for this bug?

minimal code:

from aiohttp import web


async def handle(request):
    return web.Response(text='')

app = web.Application()
app.add_routes([web.get('/', handle)])

web.run_app(app)

command: gunicorn aio.simple_server:app --worker-class aiohttp.GunicornWebWorker --workers 2

Error:

$ gunicorn aio.simple_server:app  --worker-class aiohttp.GunicornWebWorker --workers 2
[2019-02-22 23:26:32 +0200] [30784] [INFO] Starting gunicorn 19.9.0
[2019-02-22 23:26:32 +0200] [30784] [INFO] Listening at: http://127.0.0.1:8000 (30784)
[2019-02-22 23:26:32 +0200] [30784] [INFO] Using worker: aiohttp.GunicornWebWorker
[2019-02-22 23:26:32 +0200] [30787] [INFO] Booting worker with pid: 30787
======== Running on http://0.0.0.0:8080 ========
(Press CTRL+C to quit)
[2019-02-22 23:26:32 +0200] [30788] [INFO] Booting worker with pid: 30788
[2019-02-22 23:26:32 +0200] [30788] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/aiohttp/worker.py", line 52, in init_process
    super().init_process()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/dev/aio/simple_server.py", line 10, in <module>
    web.run_app(app)
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/aiohttp/web.py", line 415, in run_app
    reuse_port=reuse_port))
  File "/miniconda3/envs/aio/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/aiohttp/web.py", line 341, in _run_app
    await site.start()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/aiohttp/web_runner.py", line 102, in start
    reuse_port=self._reuse_port)
  File "/miniconda3/envs/aio/lib/python3.7/asyncio/base_events.py", line 1378, in create_server
    % (sa, err.strerror.lower())) from None
OSError: [Errno 48] error while attempting to bind on address ('0.0.0.0', 8080): address already in use
[2019-02-22 23:26:32 +0200] [30788] [INFO] Worker exiting (pid: 30788)
[2019-02-22 23:26:32 +0200] [30787] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/aiohttp/worker.py", line 52, in init_process
    super().init_process()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.run()
  File "/miniconda3/envs/aio/lib/python3.7/site-packages/aiohttp/worker.py", line 55, in run
    self._task = self.loop.create_task(self._run())
  File "/miniconda3/envs/aio/lib/python3.7/asyncio/base_events.py", line 403, in create_task
    self._check_closed()
  File "/miniconda3/envs/aio/lib/python3.7/asyncio/base_events.py", line 480, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
[2019-02-22 23:26:32 +0200] [30787] [INFO] Worker exiting (pid: 30787)
sys:1: RuntimeWarning: coroutine 'GunicornWebWorker._run' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
[2019-02-22 23:26:32 +0200] [30784] [INFO] Shutting down: Master
[2019-02-22 23:26:32 +0200] [30784] [INFO] Reason: Worker failed to boot.

Answer 1

Remove this:

web.run_app(app)

That is for running stand alone aiohttp. Assuming your code is saved as my_app.py you can run a single process with:

python my_app.py

Commenting out the run_app you can run with gunicorn:

gunicorn my_app:app  --worker-class aiohttp.GunicornWebWorker --workers 2