Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Replacing SQLite database while accessing it

I am completely new to SQLite and I intend to use it in a M2M / client-server environment where a database is generated on the server, sent to the client as a file and used on the client for data lookup.

The question is: can I replace the whole database file while the client is using it at the same time?

The question may sound silly but the client is a Linux thin client and to replace the database file a temporary file would be renamed to the final file name. In Linux, a program which has still open the older version of the file will still access the older data since the old file is preserved by the OS until all file handles have been closed. Only new open()s will access the new version of the file.

So, in short:

  • client randomly accesses the SQLite database
  • a new version of the database is received from the server and written to a temporary file
  • the temporary file is renamed to the SQLite database file

I know it is a very specific question, but maybe someone can tell me if this would be a problem for SQLite or if there are similar methods to replace a database while the client is running. I do not want to send a bunch of SQL statements from the server to the client to update the database.

like image 786
Udo G Avatar asked Apr 04 '11 08:04

Udo G


People also ask

How do I connect to an existing SQLite database?

Use the connect() method To establish a connection to SQLite, you need to pass the database name you want to connect. If you specify the database file name that already presents on the disk, it will connect to it. But if your specified SQLite database file doesn't exist, SQLite creates a new database for you.

How do I copy data from one SQLite database to another?

You'll have to attach Database X with Database Y using the ATTACH command, then run the appropriate Insert Into commands for the tables you want to transfer. You need to create the table first! If you get a "(source) database locked" error when trying to detach it, you can use COMMIT; then detach it.

Can multiple programs access the same SQLite database?

From the SQLite FAQ: Multiple processes can have the same database open at the same time. Multiple processes can be doing a SELECT at the same time. But only one process can be making changes to the database at any moment in time, however.

When should you not use SQLite?

SQLite will normally work fine as the database backend to a website. But if the website is write-intensive or is so busy that it requires multiple servers, then consider using an enterprise-class client/server database engine instead of SQLite.


1 Answers

No, you cannot just replace an open SQLite3 DB file. SQLite will keep using the same file descriptor (or handle in Windows-speak), unless you close and re-open your database. More specifically:

  • Deleting and replacing an open file is either useless (Linux) or impossible (Windows). SQLite will never get to see the contents of the new file at all.

  • Overwriting an SQLite3 DB file is a recipe for data corruption. From the SQLite3 documentation:

    Likewise, if a rogue process opens a database file or journal and writes malformed data into the middle of it, then the database will become corrupt.

    Arbitrarily overwriting the contents of the DB file can cause a whole pile of issues:

    • If you are very lucky it will just cause DB errors, forcing you to reopen the database anyway.
    • Depending on how you use the data, your application might just crash and burn.
    • Your application may try to apply an existing journal on the new file. Sounds painful? It is!
    • If you are really unlucky, the user will just get back invalid results from any queries.

The best way to deal with this would be a proper client-server implementation where the client DB file is updated from data coming from the server. In the long run that would allow for far more flexibility, while also reducing the bandwidth requirements by sending updates, rather than the whole file.

If that is not possible, you should update the client DB file in three discrete steps:

  • Send a message to the client application to close the DB. This allows the application to commit any changes, remove any journal files and clean-up its internal state.
  • Replace/Overwrite the file.
  • Send a message to the client application to re-open the DB. You would have to setup all prepared statements again, though.

If you do not want to close the DB file for some reason, then you should have your application - or even a separate process - update the original DB file using the new file as input. The SQLite3 backup API might be of interest to you in that case.

like image 87
thkala Avatar answered Sep 20 '22 10:09

thkala