Python moving files and directories from one folder to another

Question

I would like to move in python files and directories in one directory to another directory with overwrite ability.

I started with the following code:

        #moving files from progs
        path = tempfolder + 'progs/'
        for dirs,files in os.listdir(path):
                 for f in files:
                        shutil.move(os.path.join(path, f) , os.path.join(compteurfolder, f))

So for the moment, I only try to move the files and I get the following errors:

    for dirs,files in os.listdir(path):
ValueError: too many values to unpack

I guess this is because I have dirs and files but then how will I move directories too? And how to make sure it can overwrite the files in the other folder?

Hope you can help.

Answer 1

The reason this fails:

for dirs,files in os.listdir(path):

… is that os.listdir just returns a list of filenames. So, each element is a string, and you're trying to unpack that string into two variables. Compare this:

a, b = (1, 2, 3, 4, 5, 6, 7, 8)
for a, b in [(1, 2, 3, 4, 5, 6, 7, 8), (9, 10)]
dirs, files = "spam.txt"
for dirs, files in ["spam.txt", "eggs.dat"]

It's the exact same error in every case—you can't fit 8 things into 2 variables.

Meanwhile, if listdir is just returning filenames, how do you know ones are the names of regular files, are which are the names of directories? You have to ask—e.g., by using isdir.

So:

for filename in os.listdir(path):
    if os.path.isdir(filename):
        # do directory stuff
    else:
        # do regular file stuff

(But note that this can still be confusing if you have symlinks…)

---

Meanwhile, what does "do regular file stuff" mean?

Well, assuming you don't have a directory (or a symlink to a directory) with the same name as the file you're trying to move there, as the docs for shutil.move say, os.rename or shutil.copy2 will be used. If you're not on Windows, this is perfect—if you have permission to overwrite the target you will, otherwise you'll get a permissions error. But if you are on Windows, os.rename will fail if the target already exists.

If you're on 3.3 or later, you can solve this by copying the shutil.move source* and using os.replace instead, as the rename docs imply. Otherwise, you will have to delete the target before renaming the source.

_{* Some stdlib modules—including shutil—are meant to serve as sample code as well as usable helpers. In those cases, at the top of the module's docs, there will be a Source code: link.}

---

And what about "do directory stuff"? Well, if you move a directory spam to a target eggs/spam, and eggs/spam already exists as a directory, you're going to end up moving to eggs/spam/spam.

This shouldn't be surprising, as it's exactly the same thing mv on Unix and move on Windows do, which is what shutil is trying to simulate.

So, what you need to do here is delete the target (this time using shutil.rmtree) before moving the source.

---

Which means the simplest thing to do may be to not distinguish files and directories, Windows and Unix, or anything else; just do this:

for filename in os.listdir(path):
    target = os.path.join(compteurfolder, filename)
    try:
        shutil.rmtree(target)
    except NotADirectoryError:
        try:
            os.unlink(target)
        except FileNotFoundError:
            pass
    shutil.move(os.path.join(path, filename), target)