Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

SQLAlchemy occasionally erroneously returns an empty result

SQLAlchemy v1.0.6 
cx_Oracle v5.2

We've been having an issue on our production code for a little while and finally narrowed it down to the data coming back from SQLAlchemy.

Running the same query multiple times will sometimes return an empty result. Under some conditions, we can get it to return an empty result every time the code is executed. This is despite the fact that the data in the database hasn't changed at all and that pure SQL versions of the same query running directly on cx_Oracle always return the correct result.

Here is the Declarative code for SQLAlchemy:

class Database:
    def __init__(self, service_name, database, username, password):
        """
        service_name (str): The service name as defined in tnsnames.ora.
        database (str): The database within the chosen service.
        """
        self.engine = create_engine(
            r'oracle+cx_oracle://{username}:{password}@{service_name}'.format(username=username, password=password,
                                                                              service_name=service_name),
            case_sensitive=False)
        self.session_maker = sessionmaker(bind=self.engine, autoflush=False, autocommit=False)

        # Database name must be injected into every table definition; this is why tables must be procedurally generated.
        self.Base = declarative_base()  # base class for all database tables
        self.build_tables(database)

    def make_session(self):
        """Create a read-only session for the database."""
        def readonly_abort():
            raise Exception('writing is prohibited; db is read-only')

        session = self.session_maker()
        session.flush = readonly_abort
        return session

    def build_tables(self, database):
        class Lot(self.Base):
            __tablename__ = 'lot'
            __table_args__ = {'schema': database}
            lot_key = Column(Integer, primary_key=True)
            lot_id = Column(String, name='lot_id')

        self.lot = Lot

And here is the test code:

def sqlalchemy_test():
    db = dp_orm.Database(service_name, database)
    session = db.make_session()
    cursor = session.query(db.lot)
    results = cursor.first()
    if results is None:
        raise Exception

def cx_oracle_test():
    import cx_Oracle
    import set_environment_variables
    conn = cx_Oracle.Connection(username, password, service_name)
    cursor = conn.cursor()

    c = cursor.execute('SELECT * FROM {}.lot WHERE rownum <= 1'.format(database))
    results = list(c)
    if len(results) != 1:
        raise Exception

The first function, sqlalchemy_test, will error about 50% of the time. The second function, cx_oracle_test, has not yet errored. Now here's what's interesting: the problem disappears if we introduce a pause for several seconds between cursor = session.query(db.lot) and results = cursor.first(). So it looks like some sort of timing issue.

Any clue what's going on here?

EDIT: I've simplified the code necessary to create the error. Here it is:

from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

# Fix environment variables
import os
try:
    del os.environ['ORACLE_HOME']
except KeyError:
    pass
os.environ['TNS_ADMIN'] = r'C:\product\11.1.0\client_1\network\admin'
os.environ['PATH'] = r'C:\product\11.1.0\client_1\BIN;' + os.environ['PATH']

engine = create_engine(r'oracle+cx_oracle://{username}:{password}@{service_name}'.format(username='USER', password='PASSWORD', service_name='SERVICE'))
session_maker = sessionmaker(bind=engine)
base_class = declarative_base()

class Lot(base_class):
    __tablename__ = 'lot'
    __table_args__ = {'schema': 'SCHEMA_NAME'}
    lot_key = Column(Integer, primary_key=True)
    lot_id = Column(String)

session = session_maker()
cursor = session.query(Lot)
result = cursor.first()
if result is None:
    raise Exception
like image 625
Vijchti Avatar asked Sep 28 '22 08:09

Vijchti


1 Answers

Answer found: the problem was cx_Oracle 5.2. Reinstalling cx_Oracle 5.1.3 resolved the issue.

like image 75
Vijchti Avatar answered Oct 31 '22 17:10

Vijchti