directory monitoring

Question

What is the best way for me to check for new files added to a directory, I dont think the filesystemwatcher would be suitable as this is not an always on service but a method that runs when my program starts up.

there are over 20,000 files in the folder structure I am monitoring, at present I am checking each file individually to see if the filepath is in my database table, however this is taking around ten minutes and I would like to speed it up is possible,

I can store the date the folder was last checked - is it easy to get all files with createddate > last checked date.

anyone got any Ideas?

Thanks

Mark

Answer 1

Your approach is the only feasible (i.e. file system watcher allows you to see changes, not check on start).

Find out what takes so long. 20.000 checks should not take 10 minutes - maybe 1 maximum. Your program is written slowly. How do you test it?

Hint: do not ask the database, get a list of all files into memory, a list of all filesi n the database, check in memory. 20.000 SQL statements to the database are too slow, this way you need ONE to get the list.