Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

os.walk stop looking on subdirectories after first finding

I need to get the first appearance of the repository.config files in a directory and stop looking in the subdirectories.

Here is my directory tree:

./WAS80/base/disk1/ad/repository.config
./WAS80/base/disk1/md/repository.config
./WAS80/base/disk2/ad/repository.config
./WAS80/base/disk3/ad/repository.config
./WAS80/base/disk4/ad/repository.config
./WAS80/base/repository.config
./WAS80/fixpack/fp5/repository.config
./WAS80/fixpack_suplements/fp5/repository.config
./WAS80/supplements/disk1/ad/repository.config
./WAS80/supplements/disk1/md/repository.config
./WAS80/supplements/disk2/ad/repository.config
./WAS80/supplements/disk3/ad/repository.config
./WAS80/supplements/disk4/ad/repository.config
./WAS80/supplements/repository.config

I need the ones in bold and stop looking in the subdirectories.

I started tinkering with this code, but I couldn't figure it out.

pattern='repository.config'
path='/opt/was_binaries'

    def find_all(name, path):
            result = []
            for root, dirs, files in os.walk(path):
                    if name in files:
                            result.append(os.path.join(root, name))
                            continue

            return result
like image 257
radicaled Avatar asked Apr 25 '17 18:04

radicaled


2 Answers

this should do what you want:

import os

res = []

for here, dirs, files in os.walk(startdir, topdown=True):
    if 'repository.config' in files:
        res.append(os.path.join(here, 'repository.config'))
        dirs[:] = []

print(res)

whenever you encounter a 'repository.config' file, set dirs to [] in order to prevent os.walk from descending further into that directory tree.

note: it is vital for this to work to change the dirs in-place (i.e. dirs[:] = []) as opposed to rebind it (dirs = []).,

like image 137
hiro protagonist Avatar answered Nov 03 '22 16:11

hiro protagonist


First, you have to make sure that topdown is set to True (this is default) so parent directories are scanned before child directories.

Create an existing set() to remember which directories you traversed when successfully found a config file.

Then, when you find your filename in the list:

  • check if the directory of the file isn't a child of a directory you registered
  • if it's not, just note down the path of the file in existing (add os.sep, so you don't match substrings of directories starting with the current dirname at the same level: ex: path\to\dir2 should be scanned even if path\to\dir is already in the set. But path\to\dir\subdir will be successfully filtered out).

code:

import os

existing = set()
for root,dirs,files in os.walk(path,topdown=True):
    if any(root.startswith(r) for r in existing):
        # current directory is longest and contains a previously added directory: skip
        continue
    if "repository.config" in files:
        # ok, we note down root dir (+ os.sep to avoid filtering siblings) and print the result
        existing.add(root+os.sep)
        print(os.path.join(root,"repository.config"))
like image 23
Jean-François Fabre Avatar answered Nov 03 '22 18:11

Jean-François Fabre