What is the best practice for working with files and paths? [closed]

Question

I am creating a python script that essentially takes a 'path to a file' as an argument by the user. It does some post processing and creates a new file in the same directory as the original file.

For example: myscript.py C:\\A\\sub_A\\work_on_this_file.csv

I'm using the path I received itself to create a File Handler for
C:\\A\\sub_A\\final_file.csv

I've been told to use os.chdir() to navigate to the folder instead and create my final file there instead of using paths directly. What is the best practice in such a scenario? Is there a risk in not changing the working directory?

Answer 1

I would encourage you to always use absolute paths, in practice that's the most straightforward way. So, directly creating a file (or opening an existing one, doesn't matter) using the absolute path is fine.

When you are not sure whether you will have an absolute or relative path I woul suggest taking the script's directory as the base folder and then generating an absolute path, like so:

import os

cwd = os.path.abspath(os.path.dirname(__file__))
given_path = "../../myfile.csv"
csv_path = os.path.abspath(os.path.join(cwd, given_path))

Instead of __file__ use sys.argv[0] when dealing with modules/imported scripts. IMO changing the CWD is normally not required and will most likely break other things soon or later.

Answer 2

My five cents there:

• Use CSV File Reading and Writing to work with CSV Files
• Use with statement upon opening files, in that case file would always be closed in case of any unexpected error.
• Always use os.sep in order to have cross platform paths.
• Use os.path.join to form file path correctly.
• Use os.linesep when it's possible, to process file line by line correctly.