How do I distribute my pip package with data files correctly?

Question

I have a following package to distribute:

mymodule/
    data/
        1.txt
    mymodule/
        __init__.py
    tests/
        test_mymodule.py
    setup.py

In order to install it under a vitualenv I apply this command:

pip install .

Everything is installed fine but the path to my data file becomes broken.

>>> from mymodule import get
>>> print(get())
...
FileNotFoundError: [Errno 2] No such file or directory: '/home/alexander/Stuff/pip_example/mymodule_test/venv/lib/python3.5/site-packages/mymodule/../data/1.txt'

I made a research and found, the folder data was create in the root of the virtualenv's folder that caused the error. How should I improve my code to keep my tests working and to have the correct path to the data file?

The contents of the files:

data/1.txt

yes

mymodule/__init__.py

import os

src = os.path.join(
    os.path.dirname(__file__),
    '../data/1.txt'
)

def get():
    with open(src) as f:
        return f.read().strip()

tests/test_mymodule.py

import unittest
import mymodule

class MymoduleTest(unittest.TestCase):
    def test_get(self):
        s = mymodule.get()
        self.assertEqual(s, "yes")

setup.py

from distutils.core import setup

data_files = [
    ('data', ['data/1.txt'])
]

setup(
    name='mymodule',
    version='0.0',
    packages=['mymodule'],
    data_files=data_files,
)

I am new in packaging Python modules. Please help me with this issue.

=====================================================================

I figured out that I need to use sys.prefix to access the virtualenv's root. In other words, if I edit mymodule.__init__.py this way everything will work correctly:

import os
import sys

src = os.path.join(sys.prefix, 'data/1.txt')

def get():
    with open(src) as f:
        return f.read().strip()

But after that the test fails with the error:

FileNotFoundError: [Errno 2] No such file or directory: '/usr/data/1.txt'

This is because sys.prefix is /usr without activated virtualenv. So I need a different way to improve the package, I guess.

Answer 1

1. Check that your file is properly distributed when installing the package.

2. sys.prefix will not locate "your" package. The __file__ attribute of the module points to the __init__.py file. You can use this for the base path as in:

   import os
   import mymodule
   
   src = os.path.join(os.dirname(mymodule.__file__), 'data/1.txt')
   
   def get():
       with open(src) as f:
           return f.read().strip()