How to manually mark a Celery task as done and set its result?

Question

I have this Celery task:

@app.task
def do_something(with_this):

    # instantiate a class from a third party library
    instance = SomeClass()

    # this class uses callbacks to send progress info about
    # the status and progress of what we're doing
    def progress_callback(data):
        # this status will change to 'finished' later
        # but the return value that I want as the task result won't be returned
        # so this is where I should mark the task as done manually
        if data['status'] == 'working':
            # I create a custom state for this task
            do_something.update_state(
                state = 'PROGRESS',
                meta = data['progress']
            )

    # adding the callback to the instance
    instance.add_callback(progress_callback)

    # use the instance to do what I want
    # this functions returns a value that I don't want as the task result
    # so leaving this function without a return statement will make it None
    instance.do_job(with_this)

How can I mark a task as done manually ?

In this case the function reaches the end without any return statement so the task.result I get is None, I want to set the data passed to the callback function as the result and mark the task as done.

I tried using:

app.backend.mark_as_done(do_something.request.id, data)

It's successfully setting the state and the result of the task but later the result is set to the return value of the function which is here None.

Answer 1

I finally found the solution which is storing the task state and result then ignoring the task by raising an Ignore exception, for example:

from celery.exceptions import Ignore

@app.task
def do_something(with_this):

    # store the state and result manually
    # the SUCCESS state is set by this method
    app.backend.mark_as_done(
        do_something.request.id,
        the_data_to_store
    )

    # we can also use update_state which calls
    # backend.store_result just like mark_as_done
    # but we have to set the state in this case
    do_something.update_state(
        state = celery.states.SUCCESS,
        meta = the_data_to_store
    )

    # ignore the task so no other state is recorded
    # like what was happening with my function in the question
    # the task will still be acknowledged
    raise Ignore()

This is helpful when you cannot return the data that you want to store as the result.