File.lastModified() painfully slow!

Question

I'm doing a recursive copy of files and like xcopy /D I only want to copy newer files destination files (I cannot use xcopy directly since I need to alter some files in the copy process).

In java I use lastModified() to check if the destination file is older than the source file and it's very slow.

• Can I speed up the process (maybe using JNI??)?
• Are there any other copy scripts that can do the job better (copy new files + regexp change some text files)?

Copying files anyways is not an option since that will take more time than checking last modified date (copying over the network).

Answer 1

You need to determine why it is so slow.

When you are running the progrma what is the CPU utilisation of your process. If it more than 50% user, then you should be able to optmise your program, if its less than 20% there isn't so much you can do.

Usually this method is slow because the file you are examining is on disk rather than in memory. If this is the case you need to speed up how you access your disk, or get a faster drive. e.g. SSD can be 10-100x faster at doing this.

A bulk query might help. You can do this by using multiple threads to check the lastModified date. e.g. have a fixed size thread pool and add a task for each file. The size of the thread pool determines the number of files polled at once.

This allows the OS to re-order the requests to suit the layout on the disk. Note: This is fine in theory, but you have to test whether this makes things faster on your OS/hardware as its just as likely to make things slower. ;)