os.walk to crawl through folder structure

Question

I have some code that looks at a single folder and pulls out files. but now the folder structure has changed and i need to trawl throught the folders looking for files that match.

what the old code looks like

GSB_FOLDER = r'D:\Games\Gratuitous Space Battles Beta' 

def get_module_data():
    module_folder = os.path.join(GSB_FOLDER, 'data', 'modules')

    filenames = [os.path.join(module_folder, f) for f in
                  os.listdir(module_folder)]

    data = [parse_file(f) for f in filenames]

    return data

But now the folder structure has changed to be like this

• GSB_FOLDER\data\modules
  • \folder1\data\modules
  • \folder2\data\modules
  • \folder3\data\modules

where folder1,2 or 3, could be any text string

how do i rewrite the code above to do this... I have been told about os.walk but I'm just learning Python... so any help appreciated

Answer 1

Nothing much changes you just call os.walk and it will recursively go thru the directory and return files e.g.

for root, dirs, files in os.walk('/tmp'):
    if os.path.basename(root) != 'modules':
        continue
    data = [parse_file(os.path.join(root,f)) for f in files]

Here I am checking files only in folders named 'modules' you can change that check to do something else, e.g. paths which have module somewhere root.find('/modules') >= 0