How to prevent user from withdrawing funds twice?

Question

I have an endpoint in my api, which lets users specify the amount of money they want to withdraw. Before I send the withdrawal request to the payment processor, I check that the requested amount is <= the user's balance. Once the payment is processed, I deduct the amount from the user's balance.

But I'm thinking, someone could send a second request before the first payment is processed, effectively withdrawing the amount twice. How do I prevent this situation?

PS: I'm using Flask Restless and Postgres, if that makes any difference.

Answer 1

In your case, where you're co-ordinating with an external service, it's harder than you'd expect.

Traditional: prepared transactions and XA

The standard solution to this is to use two-phase commit, creating a distributed transaction, where you update the user's record before sending the payment request:

UPDATE account 
SET balance = balance - :requested_amount 
WHERE balance >= :requested_amount AND user_id = :userid`

If the update succeeds (i.e. they had enough money), you PREPARE TRANSACTION to get the DB to confirm the tx will be saved even if the DB crashes. You then send the request off to the provider, and COMMIT PREPARED or ROLLBACK PREPARED depending on the result.

A lock is held on the balance record by the prepared transaction so no other tx can begin until the prepared tx is rolled back or committed, at which point the new balance is visible.

The old balance shows up to other transactions until the prepared transaction commits or rolls back, unless they use SELECT ... FOR UPDATE or SELECT ... FOR SHARE, in which case they'll wait until the prepared TX commits/rolls back. The NOWAIT option lets them error out instead. It's all very convenient.

However, this approach scales poorly for very large client counts, and it can become problematic if the payment processor is slow or becomes unresponsive. At least in PostgreSQL there's a limit on how many prepared transactions you can have at a time.

More scalable: Keep an open-transaction log in the app

If you don't want to use two-phase commit, you'll need to keep an open transaction log instead.

Rather than just checking the users' balance, the app inserts a row into the active_transactions table as part of beginning a transaction for the user. If the user already has an active transaction, you'll need a unique constraint on active_transactions.user_id so if there are concurrent inserts all but one gets rejected.

You'll probably want to update the user's balance in the same transaction.

Other approaches, like SELECTing to check for the user before inserting a record, are unsafe and prone to race conditions. They're OK if they help provide nicer error messages, etc, but are only acceptable as additional checks.

Then you send the payment request and wait for a response. Whether it's successful or not, when you get a response you delete the open transaction journal entry and copy it to a history table; if the payment failed, you also put the user's balance back up in the same transaction, then commit. Do whatever record-keeping etc you need to in the same transaction you process the payment response in.

Either way: Transaction cleanup

With prepared transactions or an app-defined transaction journal, now you're left with the problem of what to do when your app/server crashes with transactions active and you don't know what the payment processor response for them was ... or whether you actually sent the request yet.

Most payment processor APIs offer some help for this by letting you attach application-defined tokens to each request. If you were using prepared transactions you'd use the prepared transaction Id for this; if you were doing your own transaction journal you'd use the ID you generated when you inserted the entry into your transaction journal. On restart after a crash/restart you can then check each open transaction in your app and ask the payment processor if it knows about it and if so, whether it was successful or not.

You also have to deal with cases where there was no crash, but a payment processor request or response got lost due a transient network issue, etc. You'll need code that periodically checks for apparently abandoned open transactions and re-checks them with the payment processor, like after crash recovery.

There are a number of failure modes you have to deal with:

• App crash / network issue / etc after local payment request saved, but before request sent successfully to processor

• Processor down/unreachable

• Processor sends reply (payment failed / payment OK) but your app is down/unreachable and you never get the response.

• App sends payment request then is restarted before payment processor finishes processing the request (or finishes receiving it). Cleanup code thinks the processor never received the request and discards the local transaction record, then payment processor responds to confirm the payment. (There are a few ways to deal with this, but it's really out of scope for this answer.)

• ... more?

Fun times, eh?

A useful additional sanity check is to periodically (say, daily) fetch the list of transactions from the provider, and compare it to the transactions you thought you did, making sure the completion statuses match. Flag any mismatches for human evaluation. It shouldn't happen, but ....