Is there a faster alternative to Ruby's Dir.glob?

Question

I'm using Dir.glob to visit the set of all files matching a wildcard pattern.

Dir.glob( '**/*.txt' ) { |file_name|
    parse file_name
}

Because this glob call is recursive and because lots of files are involved, glob takes a long time to build the array of files before the block starts.

What I want instead is a way of visiting all of the same files, but calling the block immediately after Ruby "discovers" each file, so that the first file is processed right away rather than after waiting for the whole directory tree to finish being searched.

Is there such a construction?

Answer 1

You can also use find and IO.popen

IO.popen("find . -name '*.txt'").each { |path| parse(path.chomp) }

Answer 2

It seems no built-in way can do this.

Hope this may help you. Find files by expanding pattern recursively (Ruby 1.9.3):

class Dir
   def self.glob_recursively( pattern, &block )
     begin
       glob(pattern, &block)
       dirs = glob('*').select { |f| File.directory? f }
       dirs.each do |dir|
         # Do not process symlink
         next if File.symlink? dir
         chdir dir
         glob_recursively(pattern, &block)
         chdir '..'
       end
     rescue SystemCallError => e
       # STDERR
       warn "ERROR: #{pwd} - #{e}"
     end
   end
end

Dir.glob_recursively ('*.txt') {|file| puts file}