Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python 3.6 glob include hidden files and folders

I try to loop over all files matching a certain extension, including those inside hidden folders. So far I haven't found a way to do this with iglob. This works for all folder except those starting with a dot:

import glob
for filename in glob.iglob('/path/**/*.ext', recursive=True):
    print(filename)

I have tried to add the dot as an optional character to no avail. I'd really like to use glob instead of residing to os.walk.

How to include all files/folders, even those starting with ., with glob?

like image 651
SeeDoubleYou Avatar asked Mar 01 '18 10:03

SeeDoubleYou


People also ask

How do I get all my glob files?

To use Glob() to find files recursively, you need Python 3.5+. The glob module supports the "**" directive(which is parsed only if you pass recursive flag) which tells python to look recursively in the directories.

How do you get a list of all files in a directory in Python?

To get a list of all the files and folders in a particular directory in the filesystem, use os. listdir() in legacy versions of Python or os. scandir() in Python 3. x.

How do I list hidden file within a directory?

To show hidden files, you need to include the /a:h modifier in that command. So, dir /a:h C:your-folder will do the trick. CMD also has specific commands for showing directories and folders. /a:d shows all hidden directories, and /a shows hidden folders.


1 Answers

I had this same issue and wished glob.glob had an optional parameter to include dot files. I wanted to be able to include ALL dot files in ALL directories including directories that start with dot. Its just not possible to do this with glob.glob. However I found that Python has pathlib standard module which has a glob function which operates differently, it will include dot files. The function operates a little differently, in particular it does not return a list of strings, but instead path objects. However I used the following

files=[]
file_refs = pathlib.Path(".").glob(pattern)
for file in file_refs:
    files.append(str(file))

The other noticeable difference I found was a glob pattern ending with **. This returned nothing in the pathlib version but would return all the files in the glob.glob one. To get the same results I added a line to check if the pattern ended with ** and if so then append /* to it.

The following code is a replacement for your example that include the files in directories starting with dot

import pathlib
for fileref in pathlib.Path('/path/').glob('**/*.ext'):
    filename = str(fileref)
    print(filename)
like image 121
waterjuice Avatar answered Oct 09 '22 21:10

waterjuice