Why is DocumentFile so slow, and what should I use instead?

Question

I want to create something like "PDF Viewer app". Application will search for all *.pdf files in location chosen by user. User can choose this folder by this function:

Intent intent = new Intent(Intent.ACTION_OPEN_DOCUMENT_TREE);
startActivityForResult(intent, REQUEST_CODE);

Then I get DocumentFile (folder):

@Override
public void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (resultCode == getActivity().RESULT_OK && requestCode == REQUEST_CODE) {
        Uri uriTree = data.getData();
        DocumentFile documentFile = DocumentFile.fromTreeUri(getActivity(), uriTree);
        //rest of code here
    }
}

Why I chose this method of selecting folder? Because I want to make possible to choose Secondary Storage (you know, in Android >= 5.0, you can't access Secondary Storage with Java.io.file).

Ok, so I get folder with all *.pdf as DocumentFile. Then I call:

for(DocumentFile file: documentFile.listFiles()){
    String fileNameToDisplay = file.getName();
}

And this is VERY SLOW. It takes almost 30 seconds when there are ~600 files in chosen folder. To prove it, I chose directory from External Storage (not secondary storage), and then I tried two solutions: DocumentFile and File. File version looks like it:

File f = new File(Environment.getExternalStorageDirectory()+"/pdffiles");
    for(File file: f.listFiles()){
        String fileNameToDisplay = file.getName();
    }
}

Second version works about 500x faster. There is almost no time in displaying all files on List View.

Why is DocumentFile so slow?

Answer 1

If you read the source code of TreeDocumentFile, you will find that each call to listFiles() and getName() invokes ContentResolver#query() under the hood. Like CommonsWare said, this would perform hundreds of queries, which is very inefficient.

Here is the source code of listFiles():

@Override
public DocumentFile[] listFiles() {
    final ContentResolver resolver = mContext.getContentResolver();
    final Uri childrenUri = DocumentsContract.buildChildDocumentsUriUsingTree(mUri,
            DocumentsContract.getDocumentId(mUri));
    final ArrayList<Uri> results = new ArrayList<>();
    Cursor c = null;
    try {
        c = resolver.query(childrenUri, new String[] {
                DocumentsContract.Document.COLUMN_DOCUMENT_ID }, null, null, null);
        while (c.moveToNext()) {
            final String documentId = c.getString(0);
            final Uri documentUri = DocumentsContract.buildDocumentUriUsingTree(mUri,
                    documentId);
            results.add(documentUri);
        }
    } catch (Exception e) {
        Log.w(TAG, "Failed query: " + e);
    } finally {
        closeQuietly(c);
    }
    final Uri[] result = results.toArray(new Uri[results.size()]);
    final DocumentFile[] resultFiles = new DocumentFile[result.length];
    for (int i = 0; i < result.length; i++) {
        resultFiles[i] = new TreeDocumentFile(this, mContext, result[i]);
    }
    return resultFiles;
}

In this function call, listFiles() made a query that only selects the document ID column. However, in your case you also want the file name for each file. Therefore, you can add the column COLUMN_DISPLAY_NAME to the query. This would retrieve the filename and document ID (which later you will convert it into Uri) in a single query and is much more efficient. There are also many other columns available such as file type, file size, and last modified time, which you may want to retrieve them as well.

c = resolver.query(mUri, new String[] {
        DocumentsContract.Document.COLUMN_DOCUMENT_ID,
        DocumentsContract.Document.COLUMN_DISPLAY_NAME
    }, null, null, null);

Within the while loop, retrieve the filename by

final String filename = c.getString(1);

The above modified code is able to instantly retrieve the Uri and filename of a directory with 1000+ files.

In summary, my recommendation is to avoid using DocumentFile if you are working with more than just a few files. Instead use ContentResolver#query() to retrieve the Uri and other information by selecting multiple columns in the query. For file operations, use the static methods in the DocumentsContract class by passing the appropriate Uri's.

By the way, it seems that the sortOrder parameter of ContentResolver#query() gets completely ignored in the above code snippet when tested on Android 11 and Android 9. I would manually sort the results instead of relying on the query order.

Answer 2

Why is DocumentFile so slow?

For ~600 files you are performing ~600 requests of a ContentProvider to get the display name, which means ~600 IPC transactions.

Instead, use MediaStore to query for all indexed media with the application/pdf MIME type.

Answer 3

To use DocumentsContract to obtain children documents, see https://developer.android.com/reference/android/provider/DocumentsContract.html#buildChildDocumentsUriUsingTree(android.net.Uri, java.lang.String).

The Uri returned from ACTION_OPEN_DOCUMENT_TREE is a tree document URI. Use the above method to build the Uri to query all children documents.

The root document ID can be obtained using https://developer.android.com/reference/android/provider/DocumentsContract.html#getTreeDocumentId(android.net.Uri) with the Uri returned from ACTION_OPEN_TREE_DOCUMENT.

Answer 4

In case someone comes up here still looking for a solution,
I built a wrapper over this with some pretty good performance.

You can check the wrapper & performance info. here: https://github.com/ItzNotABug/DocumentFileCompat