file walking in python

Question

So, I've got a working solution, but it's ugly and seems un-idiomatic. The problem is this:

For a directory tree, where every directory is set up to have:

• 1 .xc file
• at least 1 .x file
• any number of directories which follow the same format

and nothing else. I'd like to, given the root path and walk the tree applying xc() to the contents of .xc fies, x to the contents to .x files, and then doing the same thing to child folders' contents.

Actual code with explanation would be appreciated.

Thanks!

Answer 1

The function os.walk recursively walks through a directory tree, returning all file and subdirectory names.

So all you have to do is detect the .x and .xc extensions from the filenames and apply your functions when they do (untested code follows):

import os

for dirpath, dnames, fnames in os.walk("./"):
    for f in fnames:
        if f.endswith(".x"):
            x(os.path.join(dirpath, f))
        elif f.endswith(".xc"):
            xc(os.path.join(dirpath,f))

This assumes x and xc can be called on filenames; alternately you can read the contents first and pass that as a string to the functions.

Answer 2

import os

# your functions
def xc(contents): ....
def x(contents): ....

# store function references in a dict to dispatch, this is a common python idiom
funcMap = {'.xc': xc, '.x':x}

for dirpath, dirnames, filenames in os.walk(someDir):
    # use os.walk to iterate someDir's contents recursively. No
    # need to implement recursion yourself if stdlib does it for you
    for f in filenames:
        ext = os.path.splitext(f)[1]
        try:
            function = funcMap[ext]
        except KeyError:
             # no function to process files with extension 'ext', ignore it
             pass
        else:
            abspath = os.path.join(dirpath, f)
            with open(abspath) as f:
                function(f.read())

Answer 3

Seems like a good place to use recursion:

import os
def process_directory(dirName):
    """Recursively process all .xc and .x files in a parent directory and its
    subdirectories"""
    dirName = os.path.abspath(dirName)
    for f in os.listdir(dirName):
        f = os.path.join(dirName, f)
        baseName, ext = os.path.splitext(f)
        if ext == '.xc':
            print "Processing [", f, "]"
            xc(f)
        elif ext == '.x':
            print "Processing [", f, "]"
            x(f)
        elif os.path.isdir(f):
            print "\nDescending into directory", f
            process_directory(dirName=os.path.join(dirName, f))
        else:
            print "Not processing", f

I hope I haven't missed the point of your question.