How do you walk through the directories using python?

Question

I have a folder called notes, naturally they will be categorized into folders, and within those folders there will also be sub-folders for sub categories. Now my problem is I have a function that walks through 3 levels of sub directories:

def obtainFiles(path):       list_of_files = {}       for element in os.listdir(path):           # if the element is an html file then..           if element[-5:] == ".html":               list_of_files[element] = path + "/" + element           else: # element is a folder therefore a category               category = os.path.join(path, element)               # go through the category dir               for element_2 in os.listdir(category):                   dir_level_2 = os.path.join(path,element + "/" + element_2)                   if element_2[-5:] == ".html":                       print "- found file: " + element_2                       # add the file to the list of files                       list_of_files[element_2] = dir_level_2                   elif os.path.isdir(element_2):                       subcategory = dir_level_2                       # go through the subcategory dir                       for element_3 in os.listdir(subcategory):                           subcategory_path = subcategory + "/" + element_3                         if subcategory_path[-5:] == ".html":                             print "- found file: " + element_3                             list_of_files[element_3] = subcategory_path                         else:                             for element_4 in os.listdir(subcategory_path):                                  print "- found file:" + element_4 

Note that this is still very much a work in progress. Its very ugly in my eyes... What I am trying to achieve here is to go through all the folders and sub folders down and put all the file names in a dictionary called "list_of_files", the name as "key", and the full path as "value". The function doesn't quite work just yet, but was wondering how would one use the os.walk function to do a similar thing?

Thanks

Answer 1

Based on your short descriptions, something like this should work:

list_of_files = {} for (dirpath, dirnames, filenames) in os.walk(path):     for filename in filenames:         if filename.endswith('.html'):              list_of_files[filename] = os.sep.join([dirpath, filename]) 

Answer 2

an alternative is to use generator, building on @ig0774's code

import os def walk_through_files(path, file_extension='.html'):    for (dirpath, dirnames, filenames) in os.walk(path):       for filename in filenames:          if filename.endswith(file_extension):              yield os.path.join(dirpath, filename) 

and then

for fname in walk_through_files():     print(fname)