TransactionManagementError obscuring root exception

Question

When code is executing within a transaction.atomic block and an exception is raised, the database handler is marked as needing rollback. If, still within that transaction.atomic block a subsequent query is performed, the following error will be raised:

TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic'

At this point, the actual root error is obscured and pretty difficult to access, leaving you needing to jump into Django's transaction code.

A simple example that can result in this error:

def someview(request):
    with transaction.atomic():
       // do some things
       instance = SomeModel.objects.create(...)
       // some other db queries

@receiver(post_save, sender=SomeModel)
def non_critical_side_effect(
    sender, instance, created, raw, using, update_fields, **kwargs
):
    try:
        // some query that causes a database error
        SomeModelLog.objects.create(some_non_none_field=None)
    except IntegrityError:
        //notify ourselves, go on
        pass

How do you work out what's really going on when you hit this scenario, and how do you routinely avoid this scenario?

(Self-answer below - but genuinely interested in other's thoughts!)

Answer 1

If you're using django.db.transaction.atomic context manager and are stumped by a TransactionManagementError, you can determine the root cause by inspecting the value of exc_value when needs_rollback is set to True in django.db.transaction.Atomic.__exit__. This should be the exception which has prompted the need to rollback the transaction.

In terms of avoiding this error in the first place, I have adopted two approaches:

• avoid wrapping large blocks of code in transaction.atomic
• if you do require a larger transaction.atomic block, ensure that any parts in this block that can fail without needing the entire transaction to be rolled back are wrapped in their own sub-transactions.

My original example corrected such that the view can continue executing despite the signal handler encountering a database error would then appear:

def someview(request):
    with transaction.atomic():
       // do some things
       SomeModel.objects.create(invalid_field=123)

@receiver(post_save, sender=SomeModel)
def non_critical_side_effect(
    sender, instance, created, raw, using, update_fields, **kwargs
):
    try:
        with transaction.atomic():
            // some query that causes a database error
    except IntegrityError:
        // notify ourselves, go on
        pass