What's the correct manner for saving data on disk if the database is down, for later retry? [closed]

Question

I need to make sure that data which comes into a service (java app) and needs to go into a database is not lost if the database is down. Is there a correct API I should be using from Java to do this? Should I just save a the data to files and reread them later?

Answer 1

The best solution is to make sure that the database doesn't go down; e.g. run a hot standby. Typical database systems have ready-made support for this sort of thing. It is simply a matter of standing up more database replicas and configuring the clients to use them.

The problem with saving to a file is that you now have two sets of persistence code to maintain, and a bunch of complicated custom code to deal with detecting that the database is offline, switch to files, detecting that the database is back, and replaying the saved data into the database.

You also have to consider what happens if you lose the stuff saved to files; e.g. because of bugs / inadequate testing of your file-based persistence and recovery code.

---

The idea of using a queuing system with persistent queues is worth looking at. However, you need to be aware that the queuing system could go down too ... and that adds another potential failure point for your overall system. And besides, persistent queues require local disk space, and if that fills up (because the downstream database is down for a long time) you are back where you started.