Managing paths in Python projects

Question

Is there a more efficient way of dealing with hardcoded paths in Python? I find myself in the below situation where I'm appending strings together to land in the desired directory, but I don't want to inadvertently break code if I were to move files around. Is there a way to maybe tokenize these directory paths in a config file and then manage them from that location? If so how?

scripts_path = os.path.dirname(os.path.abspath(__file__)) + "/../scripts"

Answer 1

Where to put data files is a common problem and different applications handle it in different ways. Some ideas:

• Standardize a directory to read all files relative to or the location of a config file that specifies this, then put all files to read in there. This is used by the Apache server, nginx, Vim, and many other programs. There's actually a spec that tells you where you should put it. It's not universally followed but I like it.

For example, to customize the Vim app, put config in ~/.vimrc, and other configs in ~/.vim in Unix-like operating systems (Windows has corresponding locations)

• Pass config through environmental variables or command line arguments to the app. This approach is used by most apps designed to be run via Docker.

Example:

myapp --script-dir=../scripts or myapp --config conf.cfg or APP_NAME_DIR=/tmp myapp

• bake the data into the deployable. For Python, you can use the MANIFEST.in file to have pip package it. This approach is used by apps where the data is read-only (which sounds like your case) and it makes it easier to deploy. The Flask docs have some notes on how do to this.

You can combine some of these approaches. Maybe you want your app to look for a base dir name first in an environmental variable (like <app_name>_base_dir), and if that doesn't exist, in ~/.config/<app_name> or something. I recommend checking how some of your favorite apps solve this problem for more inspiration.

Once you have the base_dir path from somewhere, I highly recommend the pathlib library to manipulate paths. It comes with newer versions of Python and lets you do things like this:

from pathlib import Path

base_dir = Path(__file__).parent  # consider approaches above to get base_dir
script_dir = base_dir / 'scripts'
print(script_dir)
# prints: <location of this file>/scripts