How does commit_on_success handle being nested?

Question

I'm a bit confused about how I should handle transactions in a particular situation.

I've got some code that boils down to this:

from django.db import transaction

@transaction.commit_on_success
def process_post():
    #do stuff with database
    for reply in post_replies:
        process_post_reply(reply)

@transaction.commit_on_success
def process_post_reply(reply):
    #do stuff with database

I want to know what happens if a process_post_reply() fails.

How does commit_on_success handle being nested? Will it understand to commit each process_post_reply() or if one fails the whole process_post() rolls back?

Answer 1

Here's the source code of it: https://github.com/django/django/blob/1.2.4/django/db/transaction.py#L286

And enter_transaction_management is as simple as putting new transaction handling mode on the thread stack.

So, in your case, if process_post_reply() fails (i.e. exception occurs), then the transaction is rolled back in its entirety, and then the exception propagates upwards from process_post() as well but there is nothing to roll back.

And no, if one process_post_reply() fails then the whole process_post() is not being rolled back - there's no magic there, only COMMIT and ROLLBACK on the database level, which means that what gets rolled back is only what has been written to the DB after the last commited process_post_reply().

Summarizing, I think that what you need is just a single commit_on_success() around process_post, possibly supported by transaction savepoints - which unfortunately are available only in PostgreSQL backend, even though MySQL 5.x supports them as well.

EDIT 10 Apr 2012: Savepoint support for MySQL is now available in Django 1.4

EDIT 2 Jul 2014: Transaction management has been completely rewritten in Django 1.6 - https://docs.djangoproject.com/en/1.6/topics/db/transactions/ and commit_on_success has been deprecated.

Answer 2

To gain more control on the transaction management, it's good to use transaction.commit_manually():

@transaction.commit_on_success
def process_post(reply):
    do_stuff_with_database()
    for reply in post_replies:
        process_post_reply(transaction_commit_on_success=False)

def process_post_reply(reply, **kwargs):
    if kwargs.get('transaction_commit_on_success', True):
        with transaction.commit_manually():
            try:
                do_stuff_with_database()
            except Exception, e:
                transaction.rollback()
                raise e
            else:
                transaction.commit()
    else:
        do_stuff_with_database()

Here you can decide depending on circumstances, commit transaction or not.