Reading file using relative path in python project

Question

Say I have a python project that is structured as follows:

project     /data         test.csv     /package         __init__.py         module.py     main.py 

__init__.py:

from .module import test 

module.py:

import csv  with open("..data/test.csv") as f:     test = [line for line in csv.reader(f)] 

main.py:

import package  print(package.test) 

When I run main.py I get the following error:

 C:\Users\Patrick\Desktop\project>python main.py Traceback (most recent call last):   File "main.py", line 1, in <module>     import package   File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>     from .module import test   File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>     with open("../data/test.csv") as f: FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv' 

However, if I run module.py from the package directory I get no errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? I don't want to use absolute paths. What are some ways to deal with this?

Answer 1

Relative paths are relative to current working directory. If you do not your want your path to be, it must be absolute.

But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:

from pathlib import Path  path = Path(__file__).parent / "../data/test.csv" with path.open() as f:     test = list(csv.reader(f)) 

This requires python 3.4+ (for the pathlib module).

If you still need to support older versions, you can get the same result with:

import csv import os.path  my_path = os.path.abspath(os.path.dirname(__file__)) path = os.path.join(my_path, "../data/test.csv") with open(path) as f:     test = list(csv.reader(f)) 

[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]