Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Celery's pytest fixtures (celery_worker and celery_app) does not work

I'm trying to write a Celery(v. 4.2.1) integration test for my Django(v. 2.2.3) application.

There is a bunch of outdated articles about this around, but non of them seem to use stuff from the latest celery testing documentation - https://docs.celeryproject.org/en/v4.2.1/userguide/testing.html#fixtures

Seems like Celery comes with two fixtures for testing: celery_app and celery_worker which should allow to actually run worker in the background thread of your tests.

As the doc's say I've added

@pytest.fixture(scope='session')
def celery_config():
    return {
        'broker_url': 'memory://',
        'result_backend': 'rpc'
    }

into my conftest.py

I've wrapped my test function with

@pytest.mark.celery_app
@pytest.mark.celery_worker

usually I wrap my celery tasks with

from celery import shared_task
@shared_task
def blabla(...):
    pass

but I've even tried to replace it with

from myapp.celery import celery
@celery.task
def blabla():
    pass

What else... I run my celery task via apply_async providing eta argument.

Tried tons of ways but the celery fixtures do not affect how things work and the task call goes to actual redis instance and is picked by a worker in a separate process (if I run it) and hence my assert_called fails along with my efforts to access the object which are in the testing database.

This way it it does not load fixtures. enter image description here

This way it does not use specified fixtures because they should appear in the method arguments and break it by exceeding the number of arguments.

enter image description here

Thought that the Celery pytest plugin might be missing at all, but that's not true - tried to register it explicitly: enter image description here

Though the fixtures are available to pytest: enter image description here

But I've got into theis source code, added some wild prints there and I don't see them logged.

like image 517
scythargon Avatar asked Nov 27 '19 17:11

scythargon


People also ask

How do you pass arguments in Celery task?

To pass arguments to task with apply_async() you need to wrap them in a list and then pass the list as first argument, I.e. apply_async([arg1, arg2, arg3]) . See the documentation for more details and examples. Use delay() as an alternative.


1 Answers

OP here, I've figured it out and wrote an article - https://medium.com/@scythargon/how-to-use-celery-pytest-fixtures-for-celery-intergration-testing-6d61c91775d9

Main key:

@pytest.mark.usefixtures('celery_session_app')
@pytest.mark.usefixtures('celery_session_worker')
class MyTest():
    def test(self):
        assert mul.delay(4, 4).get(timeout=10) == 16
like image 165
scythargon Avatar answered Oct 14 '22 10:10

scythargon