Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

in-memory database in Python

I'm doing some queries in Python on a large database to get some stats out of the database. I want these stats to be in-memory so other programs can use them without going to a database.

I was thinking of how to structure them, and after trying to set up some complicated nested dictionaries, I realized that a good representation would be an SQL table. I don't want to store the data back into the persistent database, though. Are there any in-memory implementations of an SQL database that supports querying the data with SQL syntax?

like image 936
Claudiu Avatar asked Jun 15 '10 17:06

Claudiu


People also ask

How do I create a memory database in Python?

The Python interface does support the in-memory implementation that the SQLite3 C API offers. From the spec: You can also supply the special name :memory: to create a database in RAM. You can then proceed like you were using a regular database.

What is in memory in database?

In-memory databases are purpose-built databases that rely primarily on memory for data storage, in contrast to databases that store data on disk or SSDs. In-memory data stores are designed to enable minimal response times by eliminating the need to access disks.

Is SQLite an in-memory database?

SQLite in-memory databases are databases stored entirely in memory, not on disk. Use the special data source filename :memory: to create an in-memory database. When the connection is closed, the database is deleted.


1 Answers

SQLite3 might work. The Python interface does support the in-memory implementation that the SQLite3 C API offers.

From the spec:

You can also supply the special name :memory: to create a database in RAM.

It's also relatively cheap with transactions, depending on what you are doing. To get going, just:

import sqlite3 conn = sqlite3.connect(':memory:') 

You can then proceed like you were using a regular database.

Depending on your data - if you can get by with key/value (strings, hashes, lists, sets, sorted sets, etc) - Redis might be another option to explore (as you mentioned that you wanted to share with other programs).

like image 172
Tim Post Avatar answered Sep 20 '22 05:09

Tim Post