Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Flatten complex directory structure in Python

I want to move files from a complex directory structure to just one place. For example i have this deep hierarchy:

foo/
    foo2/
        1.jpg
    2.jpg
    ...

I want it to be:

1.jpg
2.jpg
...

My current solution:

def move(destination):
    for_removal = os.path.join(destination, '\\')
    is_in_parent = lambda x: x.find(for_removal) > -1
    with directory(destination):
        files_to_move = filter(is_in_parent,
                               glob_recursive(path='.'))
    for file in files_to_move:
        shutil.move(file, destination)

Definitions: directory and glob_recursive. Note, that my code only moves files to their common parent directory, not an arbitrary destination.

How can i move all files from a complex hierarchy to a single place succinctly and elegantly?

like image 463
Mirzhan Irkegulov Avatar asked Jul 09 '13 11:07

Mirzhan Irkegulov


People also ask

How do I flatten a folder structure?

Quick Start Select the folders you want to flatten. Select Actions > Flatten Folders... or (Alt+A,F) from the main menu to open the controlling dialog.

What does Flatten folder structure mean?

Flattening a folder means moving all the files from their various subfolders into one parent folder. You may want to do this to ease archiving, or to rearrange files that are stored in subfolders on a daily basis into a monthly wrap-up folder, for instance.

What is directory structure in Python?

Your app is stored as a directory structure that you can clone using Git. Each Package, Module and Form consists of files and directories within this structure.


2 Answers

I don't like testing the name of the file about to be moved to see if we're already in the destination directory. Instead, this solution only scans the subdirectories of the destination

import os
import itertools
import shutil


def move(destination):
    all_files = []
    for root, _dirs, files in itertools.islice(os.walk(destination), 1, None):
        for filename in files:
            all_files.append(os.path.join(root, filename))
    for filename in all_files:
        shutil.move(filename, destination)

Explanation: os.walk walks recursively the destination in a "top down" manner. whole filenames are constructed with the os.path.join(root, filename) call. Now, to prevent scanning files at the top of the destination, we just need to ignore the first element of the iteration of os.walk. To do that I use islice(iterator, 1, None). One other more explicit way would be to do this:

def move(destination):
    all_files = []
    first_loop_pass = True
    for root, _dirs, files in os.walk(destination):
        if first_loop_pass:
            first_loop_pass = False
            continue
        for filename in files:
            all_files.append(os.path.join(root, filename))
    for filename in all_files:
        shutil.move(filename, destination)
like image 124
Scout Avatar answered Oct 09 '22 11:10

Scout


this would do, it also renames files if they collide (I commented out the actual move and replaced with a copy):

import os
import sys
import string
import shutil

#Generate the file paths to traverse, or a single path if a file name was given
def getfiles(path):
    if os.path.isdir(path):
        for root, dirs, files in os.walk(path):
            for name in files:
                yield os.path.join(root, name)
    else:
        yield path

destination = "./newdir/"
fromdir = "./test/"
for f in getfiles(fromdir):
    filename = string.split(f, '/')[-1]
    if os.path.isfile(destination+filename):
        filename = f.replace(fromdir,"",1).replace("/","_")
    #os.rename(f, destination+filename)
    shutil.copy(f, destination+filename)
like image 36
DRC Avatar answered Oct 09 '22 12:10

DRC