how can i check if files exist for the given list of paths?

Question

I have a list of 50,000 paths and I need to check if a file exists against each of these paths. Right now, I am verifying each path independently like this:

public static List<String> filesExist(String baseDirectory, Iterable<String> paths) throws FileNotFoundException{
        File directory = new File(baseDirectory);
        if(!directory.exists()){
            throw new FileNotFoundException("No Directory found: " + baseDirectory );
        }else{
            if(!directory.isDirectory())
                throw new FileNotFoundException(baseDirectory + " is not a directory!");
        }

        List<String> filesNotFound = new ArrayList<String>();

        for (String path : paths) {
            if(!new File(baseDirectory + path).isFile())
                filesNotFound.add(path);
        }
        return filesNotFound;
    }

Is there a way to improve it so that I don't create 50,000 File objects ? I am also using guava. Is there any utility in there which can help me with bulk exists() method ?

Answer 1

The creation of 50,000 File objects is almost certainly not the bottleneck. The actual filesystem operations is probably what's making it slow.

I have two suggestions:

1. Before checking, sort paths by their location to make best use of filesystem caches.
2. If a sub-directory does not exist, you can automatically assume that all files and sub-directories therein don't exist either.