Using bulk_insert_mappings

Question

I am trying to do a bulk insert of a large list of dictionaries of the form:

results = [{'attribute': u'SEX', 'value_d': 0.0, 'value_s': u'M', 'sid': 1L}, 
           {'attribute': u'SEX', 'value_d': 0.0, 'value_s': u'M', 'sid': 2L}, 
           {'attribute': u'SEX', 'value_d': 0.0, 'value_s': u'M', 'sid': 3L},     
           {'attribute': u'SEX', 'value_d': 0.0, 'value_s': u'M', 'sid': 4L},
           ...
]

After reading about 'executemany' and bulk_insert_mappings, I decided to try the later, since it looked much simpler to work with.

Here are the 3 lines of code to execute this, using the naive assumption that since I have a list of dictionaries this would work out of the box:

Session = sessionmaker(bind=engine)
s = Session()
s.bulk_insert_mappings(Results,results)

My Results model is:

class Results(db.Model):
    __tablename__ = 'results'
    id = Column(Integer, primary_key=True, autoincrement=True)
    sid = Column(Integer)
    attribute = Column(String(2048))
    value_s = Column(String(2048))
    value_d = Column(Float)

No errors are thrown when I run this, but the data are not being inserted.

Not sure where to go with this now...

** EDIT **

As requested, the imports for this were:

In module where call was made:

os.environ['MPLCONFIGDIR'] = tempfile.mkdtemp()
import pandas as pd
from sqlalchemy.sql import label, distinct
from sqlalchemy.orm import joinedload_all
from .app import s, e

In flask app.py module:

import os
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base


# Define Flask app
app = Flask(__name__)
# grab main config from Flask-Appbuilder
app.config.from_pyfile('/Library/WebServer/wsgi/rest_api/cardioCatalogue/config.py')

from flask import current_app

URI = current_app.config['SQLALCHEMY_DATABASE_URI']

from threading import Lock

# Flask declarative base: defifne tables/models simultaneously
e = create_engine(URI, echo=True)
s = Session(e)

db_session = scoped_session(sessionmaker(autocommit=False,
                                         autoflush=False,
                                         bind=e))
Base = declarative_base()
Base.query = db_session.query_property()
Base.metadata.create_all(e)

Answer 1

Missing s.commit().

Worked like a charm and pretty bloody fast, too!

Answer 2

pymysql 0.7.1 have bug with executemany. Fixed version https://github.com/WorldException/PyMySQL or see changes https://github.com/PyMySQL/PyMySQL/pull/427/files