Importing csv data stored in a python module

Question

I have a scientific computing project for which I'm running an analysis and would like some data to be kept with the python module. I'd like the data to be 'importable' for use within a couple of examples.

The project folder hierarchy looks like this:

~/parent/
    setup.py
    setupegg.py
    /project
        __init__.py
        core.py
        /data
            __init__.py
            load_data.py
            somedata.csv
        /examples
            __init__.py
            ex1.py

I've run python setupegg.py develop from the project home folder so this package is importable from anywhere on my computer. This works so far.

The file load_data.py looks like this:

import pandas as pd
df = pd.read_csv("somedata.csv")

And the file ex1.py looks like this:

from test_module.data.load_data import df
def run():
    print df

But now when I run from project.examples import ex1 or from project.data import load_data I receive an IOError message saying 'somedata.csv' does not exist.

If I add this to the preamble of the data/load_data.py file

import os
print os.listdir('./')

It prints the list of files/folder in the directory that I'm working from.

Of course, if I use absolute pathnames to the data then it will load perfectly. How can I adjust the import statements so that reading of the csv files in the data folder does not use absolute pathnames?

Answer 1

If the csv is in the same directory as the module that opens it then you could try using the __file__ attribute:

import os
import pandas as pd
df = pd.read_csv(os.path.join(os.path.dirname(__file__), "somedata.csv"))