Open a sqlite3 database from an io.BytesIO stream?

Question

In Python3.4, is it possible to open an SQLite3 database from an io.BytesIO stream?

Something akin to:

with open("temp.db", "rb") as openf:
    byte_stream = io.BytesIO(openf.read())
sqlite3.connect(byte_stream)

The short story is: I have a stream (byte_stream) that is the sqlite database file. I can't do the following for security reasons (can't create an unencrypted file):

with open("temp.db", "wb") as openf:
    openf.write(byte_stream)
sqlite3.connect("temp.db")

Is there some lower-level API for sqlite3 that I haven't been able to find? I assume that sqlite3.connect simply calls open() at some point and opens the file as a byte stream anyway. I'm simply trying to skip that open() step.

Answer 1

This is not possible with Python's sqlite3 module.

If you were using APSW, you could try writing your own virtual file system.

Answer 2

It's not quite open-ing it so you can run SQL queries, but you can use https://github.com/uktrade/stream-sqlite to get at all the rows in the file.

import io
from stream_sqlite import stream_sqlite
import httpx

byte_stream = io.BytesIO(httpx.get('http://www.parlgov.org/static/stable/2020/parlgov-stable.db').content)

for table_name, pragma_table_info, rows in stream_sqlite(byte_stream, max_buffer_size=1_048_576):
    for row in rows:
        print(row)

(Disclaimer: I was heavily involved in the development of stream-sqlite)