What is a thread-specific os.chdir and mkdir in python?

Question

I have several threads each of which is changing working directories and occasionally creating new ones in specific working directories, copying/moving files etc in these directories. Think e.g.:

def thread1:
  while True:
    os.chdir('dir')
    os.mkdir('newdir')
    os.system('mv *.png newdir/')
    do something

def thread2:
  while True:
    os.chdir('another-dir')
    os.mkdir('another-newdir')
    os.system('mv *.png another-newdir/')
    do something

I've read that chdir, mkdir functions are not specific to threads but global. What is a way to accomplish this? I can try to use absolute paths but is that the best solution?

Answer 1

The Current Working Directory at OS level usually is process specific, not per thread. (Linux: see unshare)

Python internal file operations mostly can be re-written to use a os.path.join() and avoid chdir().

When this is not possible, e.g. for unchangeable code, for RExec kind of sandbox executions or so, then you could provide a virtual os module, open functions etc. which do path adjustment behind the scenes.

For calling external programs, you can provide a cwd=... argument in subprocess.Popen() calls and friends.

Answer 2

The working directory is an information of the process, hence all threads share the same working directory. You must work with absolute paths if you want to use multiple threads.

You can obtain absolute paths quite easily using the os.path module.

An other thing that you may consider is using python's standard library to do the copying instead of calling external processes.

See for example:

• shutil
• glob