Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS Lambda S3 copy to Postgres succeeds but doesn't copy data

I have a lambda function that uses psycopg2 to connect to a Postgres RDS instance that I am trying to configure such that it uses the aws_s3.table_import_from_s3 method to copy data directly from S3 to Postgres. The lambda function says it executes successfully, however when I poll the database, no data actually gets inserted. Here's my code:

import psycopg2
import os

def lambda_handler(event, context):
    print(f'Received event {str(event)}')
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    query_str = f"""
    select aws_s3.table_import_from_s3(
    'printer_mailings',
    '',
    '(format csv)',
    '{bucket}',
    '{key}',
    'us-east-1')
    ;"""

    try:
        connection = psycopg2.connect(user = os.environ["user"],
                                      password = os.environ["pwd"],
                                      host = os.environ["host"],
                                      port = os.environ["port"],
                                      database = os.environ["db"])

        cursor = connection.cursor()
        print('connected successfully')

        print(query_str)
        cursor.execute(query_str)
        print('Records added successfully')

    except (Exception, psycopg2.Error) as error :
        print ("Error while connecting to PostgreSQL", error)
    finally:
        #closing database connection.
            if(connection):
                cursor.close()
                connection.close()
                print("PostgreSQL connection is closed")

Any idea why this doesn't work? If I run the query_str in PSQL it works, but not here even though I have successfully connected to the DB.

like image 552
DBA108642 Avatar asked Aug 12 '26 05:08

DBA108642


1 Answers

    cursor = connection.cursor()
    print('connected successfully')

    print(query_str)
    cursor.execute(query_str)
    connection.commit() # you missed this
    print('Records added successfully')
like image 135
ROMMEL Avatar answered Aug 15 '26 00:08

ROMMEL



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!