How do python submodules share a scarce resource between them?

Question

I am breaking up a large monolothic python file into six separate submodules.

Originally in onebigfile.py, I had

conn = MySqldb.connect()
c = conn.cursor()

and then a function would use it as so:

def getFromDB():
     c.execute(sql)

Now, I have restructured my module to

NewModule/
  __init__.py
  users.py
  accounts.py
  sixmoreofthese.py

What I'm puzzling over is what to do with semi-scarce resources like my MySQL connection.

Is there a way to access a parent namespace, e.g. conn. and c. could be put in __init__.py ? I don't want to instantiate a whole bunch of connections to MySQL. Just dumping them there and calling them as if they're part of the global namespace doesn't work.. That is:

__init__.py:
   conn = MySqldb.connect()
   c = conn.cursor() 

> import NewModule
> NewModule.users.login('a','b')
--- login function calls the Mysql c. from the global namespace and can't find it. 

To anticipate one suggestion: it makes sense to split these files -- there's roughly 50-75k worth of python, and a group of people that need to work with the code, plus there are pretty clear conceptual groupings of functionality.

Answer 1

You can certainly share the connection object between python modules, and your idea to have the connection be a module-scoped variable will accomplish this nicely (with a few exceptions, Python modules are singletons, so even if they are imported multiple times from different files, they are only loaded once). You probably do not want to share a single cursor object between modules, especially if you are using a multi-threaded environment (e.g. running a web application in mod_wsgi or similar).

One suggestion might be to move the connection into a database-specific module, like db.py, and initialize/access it with a get_connection() method. This will allow your application to bootstrap and load any configuration it needs to connect (username, password, hostname) gracefully. This might be as simple as:

# db.py
connection = None
def get_connection():
    global connection
    if not connection:
        connection = MySqldb.connect() # possibly with configuration vars passed in
    return connection

Your other code which uses the databse could look like:

# other_module.py
import db
curs = db.get_connection().cursor()
# do stuff
curs.close()