os.walk without hidden folders

Question

I need to list all files with the containing directory path inside a folder. I tried to use os.walk, which obviously would be the perfect solution.

However, it also lists hidden folders and files. I'd like my application not to list any hidden folders or files. Is there any flag you can use to make it not yield any hidden files?

Cross-platform is not really important to me, it's ok if it only works for linux (.* pattern)

Answer 1

No, there is no option to os.walk() that'll skip those. You'll need to do so yourself (which is easy enough):

for root, dirs, files in os.walk(path):     files = [f for f in files if not f[0] == '.']     dirs[:] = [d for d in dirs if not d[0] == '.']     # use files and dirs 

Note the dirs[:] = slice assignment; os.walk recursively traverses the subdirectories listed in dirs. By replacing the elements of dirs with those that satisfy a criteria (e.g., directories whose names don't begin with .), os.walk() will not visit directories that fail to meet the criteria.

This only works if you keep the topdown keyword argument to True, from the documentation of os.walk():

When topdown is True, the caller can modify the dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into the subdirectories whose names remain in dirnames; this can be used to prune the search, impose a specific order of visiting, or even to inform walk() about directories the caller creates or renames before it resumes walk() again.

Answer 2

I realize it wasn't asked in the question, but I had a similar problem where I wanted to exclude both hidden files and files beginning with __, specifically __pycache__ directories. I landed on this question because I was trying to figure out why my list comprehension was not doing what I expected. I was not modifying the list in place with dirnames[:].

I created a list of prefixes I wanted to exclude and modified the dirnames in place like so:

    exclude_prefixes = ('__', '.')  # exclusion prefixes     for dirpath, dirnames, filenames in os.walk(node):         # exclude all dirs starting with exclude_prefixes         dirnames[:] = [dirname                        for dirname in dirnames                        if not dirname.startswith(exclude_prefixes)]