GAE/P: Transaction safety with API calls

Question

Suppose you use a transaction to process a Stripe payment and update a user entity:

@ndb.transactional
def process_payment(user_key, amount):
    user = user_key.get()
    user.stripe_payment(amount) # API call to Stripe
    user.balance += amount
    user.put()

It is possible that the Stripe API call succeeds but that the put fails because of contention. The user would then be charged, but his account wouldn't reflect the payment.

You could pull the Stripe API call out of the transaction and do the transaction afterwards, but it seems like you still have the same problem. The charge succeeds but the transaction fails and the user's account isn't credited.

This seems like a really common scenario. How does one properly handle this?

Answer 1

For proper operation the transactional function need to be idempotent. So you can't make the stripe call inside such function as it would make it non-idempotent.

I'd make the stripe call separate and, on API success, I'd call a transactional function to update the user's account balance (which can be safely retried in case of contention).

Maybe even create a separate, independent entity to reflect the stripe API call result? Such entity should have no room for contention since it's only written once - when the stripe transaction takes place. This would allow you to:

• keep a history of account transactions - pointing to these entities
• have some sanity checks looking for orphan stripe transaction (if for whatever reason the account transaction call fails even after retries) and do something about it.

@thebjorn's comment is a good one: a multi-step approach could make the process pretty solid:

• a transactional function updating the account with the intent to execute a stripe transaction, which also transactionally enqueues a push task to perform the stripe API call. The task is enqueued only if the transaction succeds
• the push queue task makes the stripe API call (eventually creating the stripe transaction entity) and, on success, calls the transactional function to update the account balance