Python Watchdog process existing files on startup

Question

I have a simple Watchdog and Queue process to monitor files in a directory. Code taken from https://camcairns.github.io/python/2017/09/06/python_watchdog_jobs_queue.html

import time
from watchdog.events import PatternMatchingEventHandler
from watchdog.observers import Observer
from queue import Queue
from threading import Thread

dir_path = "/data"

def process_queue(q):

    while True:
        if not q.empty():
            event = q.get()
            print("New event %s" % event)

        time.sleep(5)


class FileWatchdog(PatternMatchingEventHandler):

    def __init__(self, queue, patterns):
        PatternMatchingEventHandler.__init__(self, patterns=patterns)
        self.queue = queue

    def process(self, event):
        self.queue.put(event)

    def on_created(self, event):
        self.process(event)


if __name__ == '__main__':

    watchdog_queue = Queue()

    worker = Thread(target=process_queue, args=(watchdog_queue,))
    worker.setDaemon(True)
    worker.start()

    event_handler = FileWatchdog(watchdog_queue, patterns="*.ini")
    observer = Observer()
    observer.schedule(event_handler, path=dir_path)
    observer.start()

    try:
        while True:
            time.sleep(2)
    except KeyboardInterrupt:
        observer.stop()

    observer.join()

Once the process is running new files are processed correctly. However if I restart the process and a file already exists in the directory it is ignored.

I have tried to create a dict to add to the queue

    for file in os.listdir(dir_path):
        if file.endswith(".ini"):
             file_path = os.path.join(dir_path, file)
             event = {'event_type' : 'on_created', 'is_directory' : 'False', 'src_path' : file_path}
             watchdog_queue.put(event)

but it's expecting an object of type (class 'watchdog.events.FileCreatedEvent') and I can't work out how to create this.

Alternatively I can see in the Watchdog documentation (class watchdog.utils.dirsnapshot.DirectorySnapshot) but I cannot work out how to run this and add it to the queue.

Any suggestions on how I can add existing files to the queue on startup ?

Answer 1

This code should do what you are trying to achieve.

from watchdog.events import FileCreatedEvent

# Loop to get all files; dir_path is your lookup folder.

for file in os.listdir(dir_path):
    filename = os.path.join(dir_path, file)
    event = FileCreatedEvent(filename)
    watchdog_queue.put(event)