Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Rapid inserts using SQLite

I'm developing a data logging application that will be writing a row to a SQLite database roughly every 100ms. Without using transactions, inserting 100 rows at this interval takes 20 seconds - twice as long as it should.

Does SQLite provide anything that can help with what I'm trying to do, or is it a case of rolling my own solution (e.g. committing a transaction every N rows or N seconds)?

like image 845
Andrew Stephens Avatar asked Jul 31 '26 03:07

Andrew Stephens


2 Answers

Commit a transaction only when no new data value is yet available; otherwise, write the new value(s) in the same transaction.

This automatically throttles transactions to the largest sustainable rate.

like image 131
CL. Avatar answered Aug 03 '26 02:08

CL.


Have you considered 'Naive Inserts`?

http://blog.quibb.org/2010/08/fast-bulk-inserts-into-sqlite/

like image 22
Colin Avatar answered Aug 03 '26 02:08

Colin



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!