error inserting values to db with psycopg2 module

Question

I am attempting to insert a dataframe into my postgres database using the pscycopg2 module used with sqlalchemy. The process is loading an excel file into a pandas dataframe and then inserting the dataframe into the database via the predefined table schema.

I believe these are the relevant lines of code:

post_meta.reflect(schema="users")
df = pd.read_excel(path) 
table = sql.Table(table_name, post_meta, schema="users")
dict_items = df.to_dict(orient='records')
connection.execute(table.insert().values(dict_items))

I'm getting the following error:

<class 'sqlalchemy.exc.ProgrammingError'>, ProgrammingError("(psycopg2.ProgrammingError) can't adapt type 'numpy.int64'",)

All data field types in the dataframe are int64.

I can't seem to find a similar question or information regarding why this error is and what it means.

Any direction would be great. Thanks

Answer 1

Looks like you're trying to insert numpy integers, and psycopg2 doesn't know how to handle those objects. You need to convert them to normal python integers first. Maybe try calling the int() function on each value... Please provide more context with code if that fails.

Answer 2

I also ran into this error, and then realized that I was trying to insert integer data into a SqlAlchemy Numeric column, which maps to float, not int. Changing the offending DataFrame column to float did the trick for me: df[col] = df[col].astype(float)

Perhaps you are also trying to insert integer data into a non-integer column?