Is it possible to use SQLite for querying over Parquet files?
CREATE TABLE test USING parquet('test.parquet');
SELECT * from test;
Use Pandas to import the data first!
import pandas as pd
import sqlite3
parquet_path = 'my-data.parquet'
table_name = 'my_data_table'
query = f"SELECT * from {table_name}"
db_conn = sqlite3.connect(database='/tmp/my.db')
df_parquet = pd.read_parquet(parquet_path)
num_rows_inserted = df_parquet.to_sql(table_name, db_conn, index=False)
cursor = db_conn.execute(query)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With