Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas DataFrame to PostgreSQL using Psycopg2

Is it possible to write a Pandas dataframe to PostgreSQL database using psycopg2?

Endgoal is to be able to write a Pandas dataframe to Amazon RDS PostgreSQL instance.

like image 847
verkter Avatar asked Nov 29 '15 07:11

verkter


People also ask

How does psycopg2 connect to Python?

Establishing connection using python The connection class of the psycopg2 represents/handles an instance of a connection. You can create new connections using the connect() function. This accepts the basic connection parameters such as dbname, user, password, host, port and returns a connection object.

How do I load my pandas DataFrame to my database 10x faster?

Instead of uploading your pandas DataFrames to your PostgreSQL database using the pandas. to_sql() function, you can write the data to a CSV file and COPY the file into PostgreSQL, which is considerably faster, as I'll demonstrate below.


1 Answers

If you use a SQLAlchemy engine around psycopg2, you could probably use df.to_sql. See: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_sql.html

like image 68
David Maust Avatar answered Sep 28 '22 10:09

David Maust