How to check which StorageVolume we have access to, and which we don't?

Question

Background

Google (sadly) plans to ruin storage permission so that apps won't be able to access the file system using the standard File API (and file-paths). Many are against it as it changes the way apps can access the storage and in many ways it's a restricted and limited API.

As a result, we will need to use SAF (storage access framework) entirely on some future Android version (on Android Q we can, at least temporarily, use a flag to use the normal storage permission), if we wish to deal with various storage volumes and reach all files there.

So, for example, suppose you want to make a file manager and show all the storage volumes of the device, to show what the user can grant access to, and if you already have access to each, you just enter it. Such a thing seems very legitimate, but as I can't find a way to do it.

The problem

Starting from API 24 (here), we finally have the ability to list all of the storage volumes, as such:

    val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
    val storageVolumes = storageManager.storageVolumes

And, for the first time ever, we can have an Intent to request access to a storageVolume (here). So if we want, for example, to request the user to grant access to the primary one (which will just start from there, actually, and not really ask anything), we could use this:

startActivityForResult(storageManager.primaryStorageVolume.createOpenDocumentTreeIntent(), REQUEST_CODE__DIRECTORTY_PERMISSION)

Instead of startActivityForResult(Intent(Intent.ACTION_OPEN_DOCUMENT_TREE), REQUEST_CODE__DIRECTORTY_PERMISSION) , and hoping the user will choose the correct thing there.

And to finally get the access to what the user chose, we have this:

@TargetApi(Build.VERSION_CODES.KITKAT)
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
    super.onActivityResult(requestCode, resultCode, data)
    if (requestCode == REQUEST_CODE__DIRECTORTY_PERMISSION && resultCode == Activity.RESULT_OK && data != null) {
        val treeUri = data.data ?: return
        contentResolver.takePersistableUriPermission(treeUri, Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION)
        val pickedDir = DocumentFile.fromTreeUri(this, treeUri)
        ...

So far we can request for permission on the various storage volumes...

However, the problem arises if you want to know which you got permission to and which you haven't.

What I've found

1. There is a video about "Scoped Directory Access" by Google (here), which they talk specifically about the StorageVolume class. They even give information about listening to mount-events of StorageVolume, but they don't tell anything about identifying those that we got access to.

2. The only ID of StorageVolume class is uuid , but it's not even guaranteed to return anything. And indeed it returns null in various cases. For example the case of the primary storage.

3. When using the createOpenDocumentTreeIntent function, I've noticed there is a Uri hidden inside, probably telling which to start with. It's inside the extras, in a key called "android.provider.extra.INITIAL_URI". When checking its value on the primary storage, for example, I got this:

   content://com.android.externalstorage.documents/root/primary

4. When I look at the Uri I get in return in the onActivityResult, I get something a bit similar to #2, but different for the treeUri variable I've shown :

   content://com.android.externalstorage.documents/tree/primary%3A

5. In order to get the list of what you have access to so far, you can use this:

   val persistedUriPermissions = contentResolver.persistedUriPermissions

This returns you a list of UriPermission, each has a Uri. Sadly, when I use it, I get the same as on #3, which I can't really compare to what I get from StorageVolume :

content://com.android.externalstorage.documents/tree/primary%3A

So as you can see, I can't find any kind of mapping between the list of storage volumes and what the user grants.

I can't even know if the user has chosen a storage volume at all, because the function of createOpenDocumentTreeIntent only send the user to the StorageVolume, but it's still possible to select a folder instead.

The only thing that I do have, is a chunk of workaround functions I've found on other questions here, and I don't think they are reliable, especially now that we don't really have access to File API and file-path.

I've written them here, in case you think they are useful:

@TargetApi(VERSION_CODES.LOLLIPOP)
private static String getVolumeIdFromTreeUri(final Uri treeUri) {
    final String docId = DocumentsContract.getTreeDocumentId(treeUri);
    final int end = docId.indexOf(':');
    String result = end == -1 ? null : docId.substring(0, end);
    return result;
}

private static String getDocumentPathFromTreeUri(final Uri treeUri) {
    final String docId = DocumentsContract.getTreeDocumentId(treeUri);
    //TODO avoid using spliting of a string (because it uses extra strings creation)
    final String[] split = docId.split(":");
    if ((split.length >= 2) && (split[1] != null))
        return split[1];
    else
        return File.separator;
}

public static String getFullPathOfDocumentFile(Context context, DocumentFile documentFile) {
    String volumePath = getVolumePath(context, getVolumeIdFromTreeUri(documentFile.getUri()));
    if (volumePath == null)
        return null;
    DocumentFile parent = documentFile.getParentFile();
    if (parent == null)
        return volumePath;
    final LinkedList<String> fileHierarchy = new LinkedList<>();
    while (true) {
        fileHierarchy.add(0, documentFile.getName());
        documentFile = parent;
        parent = documentFile.getParentFile();
        if (parent == null)
            break;
    }
    final StringBuilder sb = new StringBuilder(volumePath).append(File.separator);
    for (String fileName : fileHierarchy)
        sb.append(fileName).append(File.separator);
    return sb.toString();
}

/**
 * Get the full path of a document from its tree URI.
 *
 * @param treeUri The tree RI.
 * @return The path (without trailing file separator).
 */
public static String getFullPathFromTreeUri(Context context, final Uri treeUri) {
    if (treeUri == null)
        return null;
    String volumePath = getVolumePath(context, getVolumeIdFromTreeUri(treeUri));
    if (volumePath == null)
        return File.separator;
    if (volumePath.endsWith(File.separator))
        volumePath = volumePath.substring(0, volumePath.length() - 1);
    String documentPath = getDocumentPathFromTreeUri(treeUri);
    if (documentPath.endsWith(File.separator))
        documentPath = documentPath.substring(0, documentPath.length() - 1);
    if (documentPath.length() > 0)
        if (documentPath.startsWith(File.separator))
            return volumePath + documentPath;
        else return volumePath + File.separator + documentPath;
    return volumePath;
}

/**
 * Get the path of a certain volume.
 *
 * @param volumeId The volume id.
 * @return The path.
 */
private static String getVolumePath(Context context, final String volumeId) {
    if (VERSION.SDK_INT < VERSION_CODES.LOLLIPOP)
        return null;
    try {
        final StorageManager storageManager = (StorageManager) context.getSystemService(Context.STORAGE_SERVICE);
        if (VERSION.SDK_INT >= VERSION_CODES.N) {
            final Class<?> storageVolumeClazz = StorageVolume.class;
            final Method getPath = storageVolumeClazz.getMethod("getPath");
            final List<StorageVolume> storageVolumes = storageManager.getStorageVolumes();
            for (final StorageVolume storageVolume : storageVolumes) {
                final String uuid = storageVolume.getUuid();
                final boolean primary = storageVolume.isPrimary();
                // primary volume?
                if (primary && PRIMARY_VOLUME_NAME.equals(volumeId)) {
                    return (String) getPath.invoke(storageVolume);
                }
                // other volumes?
                if (uuid != null && uuid.equals(volumeId))
                    return (String) getPath.invoke(storageVolume);
            }
            return null;
        }
        final Class<?> storageVolumeClazz = Class.forName("android.os.storage.StorageVolume");
        final Method getVolumeList = storageManager.getClass().getMethod("getVolumeList");
        final Method getUuid = storageVolumeClazz.getMethod("getUuid");
        //noinspection JavaReflectionMemberAccess
        final Method getPath = storageVolumeClazz.getMethod("getPath");
        final Method isPrimary = storageVolumeClazz.getMethod("isPrimary");
        final Object result = getVolumeList.invoke(storageManager);
        final int length = Array.getLength(result);
        for (int i = 0; i < length; i++) {
            final Object storageVolumeElement = Array.get(result, i);
            final String uuid = (String) getUuid.invoke(storageVolumeElement);
            final Boolean primary = (Boolean) isPrimary.invoke(storageVolumeElement);
            // primary volume?
            if (primary && PRIMARY_VOLUME_NAME.equals(volumeId)) {
                return (String) getPath.invoke(storageVolumeElement);
            }
            // other volumes?
            if (uuid != null && uuid.equals(volumeId))
                return (String) getPath.invoke(storageVolumeElement);
        }
        // not found.
        return null;
    } catch (Exception ex) {
        return null;
    }
}

The question

How can I map between the list of StorageVolume and the list of granted UriPermission ?

In other words, given a list of StorageVolume, how can I know to which I have access to and which I don't , and if I do have access, to open it and see what's inside?

Answer 1

Here is an alternate way to get what you want. It is a work-around like you have posted without using reflection or file paths.

On an emulator, I see the following items for which I have permitted access.

persistedUriPermissions array contents (value of URI only):

0 uri = content://com.android.externalstorage.documents/tree/primary%3A
1 uri = content://com.android.externalstorage.documents/tree/1D03-2E0E%3ADownload
2 uri = content://com.android.externalstorage.documents/tree/1D03-2E0E%3A
3 uri = content://com.android.externalstorage.documents/tree/primary%3ADCIM
4 uri = content://com.android.externalstorage.documents/tree/primary%3AAlarms

"%3A" is a colon (":"). So, it appears that the URI is constructed as follows for a volume where "<volume>" is the UUID of the volume.

uri = "content://com.android.externalstorage.documents/tree/<volume>:"

If the uri is a directory directly under a volume, then the structure is:

uri = "content://com.android.externalstorage.documents/tree/<volume>:<directory>"

For directories deeper in the structure, the format is:

uri = "content://com.android.externalstorage.documents/tree/<volume>:<directory>/<directory>/<directory>..."

So, it is just a matter of extracting volumes from URIs in these formats. The volume extracted can be used as a key for StorageManager.storageVolumes. The following code does just this.

It seems to me that there should be an easier way to go about this. There must be a missing linkage in the API between storage volumes and URIs. I can't say that this technique covers all circumstances.

I also question the UUID that is returned by storageVolume.uuid which seems to be a 32-bit value. I thought that UUIDs are 128 bits in length. Is this an alternative format for a UUID or somehow derived from the UUID? Interesting, and it is all about to drop! :(

MainActivity.kt

class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
        var storageVolumes = storageManager.storageVolumes
        val storageVolumePathsWeHaveAccessTo = HashSet<String>()

        checkAccessButton.setOnClickListener {
            checkAccessToStorageVolumes()
        }

        requestAccessButton.setOnClickListener {
            storageVolumes = storageManager.storageVolumes
            val primaryVolume = storageManager.primaryStorageVolume
            val intent = primaryVolume.createOpenDocumentTreeIntent()
            startActivityForResult(intent, 1)
        }
    }

    private fun checkAccessToStorageVolumes() {
        val storageVolumePathsWeHaveAccessTo = HashSet<String>()
        val persistedUriPermissions = contentResolver.persistedUriPermissions
        persistedUriPermissions.forEach {
            storageVolumePathsWeHaveAccessTo.add(it.uri.toString())
        }
        val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
        val storageVolumes = storageManager.storageVolumes

        for (storageVolume in storageVolumes) {
            val uuid = if (storageVolume.isPrimary) {
                // Primary storage doesn't get a UUID here.
                "primary"
            } else {
                storageVolume.uuid
            }
            val volumeUri = uuid?.let { buildVolumeUriFromUuid(it) }
            when {
                uuid == null -> 
                    Log.d("AppLog", "UUID is null for ${storageVolume.getDescription(this)}!")
                storageVolumePathsWeHaveAccessTo.contains(volumeUri) -> 
                    Log.d("AppLog", "Have access to $uuid")
                else -> Log.d("AppLog", "Don't have access to $uuid")
            }
        }
    }

    private fun buildVolumeUriFromUuid(uuid: String): String {
        return DocumentsContract.buildTreeDocumentUri(
            "com.android.externalstorage.documents",
            "$uuid:"
        ).toString()
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)
        Log.d("AppLog", "resultCode:$resultCode")
        val uri = data?.data ?: return
        val takeFlags =
            Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION
        contentResolver.takePersistableUriPermission(uri, takeFlags)
        Log.d("AppLog", "granted uri: ${uri.path}")
    }
}

Answer 2

EDIT: Found a workaround, but it might not work some day.

It uses reflection to get the real path of the StorageVolume instance, and it uses what I had before to get the path of persistedUriPermissions . If there are intersections between them, it means I have access to the storageVolume.

Seems to work on emulator, which finally has both internal storage and SD-card.

Hopefully we will get proper API and not need to use reflections.

If there is a better way to do it, without those kinds of tricks, please let me know.

So, here's an example:

MainActivity.kt

class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
        val storageVolumes = storageManager.storageVolumes
        val primaryVolume = storageManager.primaryStorageVolume
        checkAccessButton.setOnClickListener {
            val persistedUriPermissions = contentResolver.persistedUriPermissions
            val storageVolumePathsWeHaveAccessTo = HashSet<String>()
            Log.d("AppLog", "got access to paths:")
            for (persistedUriPermission in persistedUriPermissions) {
                val path = FileUtilEx.getFullPathFromTreeUri(this, persistedUriPermission.uri)
                        ?: continue
                Log.d("AppLog", "path: $path")
                storageVolumePathsWeHaveAccessTo.add(path)
            }
            Log.d("AppLog", "storage volumes:")
            for (storageVolume in storageVolumes) {
                val volumePath = FileUtilEx.getVolumePath(storageVolume)
                if (volumePath == null) {
                    Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - failed to get volumePath")
                } else {
                    val hasAccess = storageVolumePathsWeHaveAccessTo.contains(volumePath)
                    Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - volumePath:$volumePath - gotAccess? $hasAccess")
                }
            }
        }
        requestAccessButton.setOnClickListener {
            val intent = primaryVolume.createOpenDocumentTreeIntent()
            startActivityForResult(intent, 1)
        }
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)
        Log.d("AppLog", "resultCode:$resultCode")
        val uri = data?.data ?: return
        val takeFlags = Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION
        contentResolver.takePersistableUriPermission(uri, takeFlags)
        val fullPathFromTreeUri = FileUtilEx.getFullPathFromTreeUri(this, uri)
        Log.d("AppLog", "granted uri:$uri $fullPathFromTreeUri")
    }
}

FileUtilEx.java

/**
 * Get the full path of a document from its tree URI.
 *
 * @param treeUri The tree RI.
 * @return The path (without trailing file separator).
 */
public static String getFullPathFromTreeUri(Context context, final Uri treeUri) {
    if (treeUri == null)
        return null;
    String volumePath = getVolumePath(context, getVolumeIdFromTreeUri(treeUri));
    if (volumePath == null)
        return File.separator;
    if (volumePath.endsWith(File.separator))
        volumePath = volumePath.substring(0, volumePath.length() - 1);
    String documentPath = getDocumentPathFromTreeUri(treeUri);
    if (documentPath.endsWith(File.separator))
        documentPath = documentPath.substring(0, documentPath.length() - 1);
    if (documentPath.length() > 0)
        if (documentPath.startsWith(File.separator))
            return volumePath + documentPath;
        else return volumePath + File.separator + documentPath;
    return volumePath;
}

public static String getVolumePath(StorageVolume storageVolume){
    if (VERSION.SDK_INT < VERSION_CODES.LOLLIPOP)
        return null;
    try{
        final Class<?> storageVolumeClazz = StorageVolume.class;
        final Method getPath = storageVolumeClazz.getMethod("getPath");
        return (String) getPath.invoke(storageVolume);
    } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
        e.printStackTrace();
    }
    return null;
}

/**
 * Get the path of a certain volume.
 *
 * @param volumeId The volume id.
 * @return The path.
 */
@SuppressLint("ObsoleteSdkInt")
private static String getVolumePath(Context context, final String volumeId) {
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP)
        return null;
    try {
        final StorageManager storageManager = (StorageManager) context.getSystemService(Context.STORAGE_SERVICE);
        if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.N) {
            final Class<?> storageVolumeClazz = StorageVolume.class;
            //noinspection JavaReflectionMemberAccess
            final Method getPath = storageVolumeClazz.getMethod("getPath");
            final List<StorageVolume> storageVolumes = storageManager.getStorageVolumes();
            for (final StorageVolume storageVolume : storageVolumes) {
                final String uuid = storageVolume.getUuid();
                final boolean primary = storageVolume.isPrimary();
                // primary volume?
                if (primary && PRIMARY_VOLUME_NAME.equals(volumeId)) {
                    return (String) getPath.invoke(storageVolume);
                }
                // other volumes?
                if (uuid != null && uuid.equals(volumeId))
                    return (String) getPath.invoke(storageVolume);
            }
            return null;
        }
        final Class<?> storageVolumeClazz = Class.forName("android.os.storage.StorageVolume");
        final Method getVolumeList = storageManager.getClass().getMethod("getVolumeList");
        final Method getUuid = storageVolumeClazz.getMethod("getUuid");
        //noinspection JavaReflectionMemberAccess
        final Method getPath = storageVolumeClazz.getMethod("getPath");
        final Method isPrimary = storageVolumeClazz.getMethod("isPrimary");
        final Object result = getVolumeList.invoke(storageManager);
        final int length = Array.getLength(result);
        for (int i = 0; i < length; i++) {
            final Object storageVolumeElement = Array.get(result, i);
            final String uuid = (String) getUuid.invoke(storageVolumeElement);
            final Boolean primary = (Boolean) isPrimary.invoke(storageVolumeElement);
            // primary volume?
            if (primary && PRIMARY_VOLUME_NAME.equals(volumeId)) {
                return (String) getPath.invoke(storageVolumeElement);
            }
            // other volumes?
            if (uuid != null && uuid.equals(volumeId))
                return (String) getPath.invoke(storageVolumeElement);
        }
        // not found.
        return null;
    } catch (Exception ex) {
        return null;
    }
}

/**
 * Get the document path (relative to volume name) for a tree URI (LOLLIPOP).
 *
 * @param treeUri The tree URI.
 * @return the document path.
 */
@TargetApi(VERSION_CODES.LOLLIPOP)
private static String getDocumentPathFromTreeUri(final Uri treeUri) {
    final String docId = DocumentsContract.getTreeDocumentId(treeUri);
    //TODO avoid using spliting of a string (because it uses extra strings creation)
    final String[] split = docId.split(":");
    if ((split.length >= 2) && (split[1] != null))
        return split[1];
    else
        return File.separator;
}

/**
 * Get the volume ID from the tree URI.
 *
 * @param treeUri The tree URI.
 * @return The volume ID.
 */
@TargetApi(VERSION_CODES.LOLLIPOP)
private static String getVolumeIdFromTreeUri(final Uri treeUri) {
    final String docId = DocumentsContract.getTreeDocumentId(treeUri);
    final int end = docId.indexOf(':');
    String result = end == -1 ? null : docId.substring(0, end);
    return result;
}

activity_main.xml

<LinearLayout
  xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent"
  android:gravity="center" android:orientation="vertical" tools:context=".MainActivity">

  <Button
    android:id="@+id/checkAccessButton" android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="checkAccess"/>

  <Button
    android:id="@+id/requestAccessButton" android:layout_width="wrap_content" android:layout_height="wrap_content" android:text="requestAccess"/>

</LinearLayout>

To put it in a simple function, here:

/** for each storageVolume, tells if we have access or not, via a HashMap (true for each iff we identified it has access*/
fun getStorageVolumesAccessState(context: Context): HashMap<StorageVolume, Boolean> {
    val storageManager = context.getSystemService(Context.STORAGE_SERVICE) as StorageManager
    val storageVolumes = storageManager.storageVolumes
    val persistedUriPermissions = context.contentResolver.persistedUriPermissions
    val storageVolumePathsWeHaveAccessTo = HashSet<String>()
    //            Log.d("AppLog", "got access to paths:")
    for (persistedUriPermission in persistedUriPermissions) {
        val path = FileUtilEx.getFullPathFromTreeUri(context, persistedUriPermission.uri)
                ?: continue
        //                Log.d("AppLog", "path: $path")
        storageVolumePathsWeHaveAccessTo.add(path)
    }
    //            Log.d("AppLog", "storage volumes:")
    val result = HashMap<StorageVolume, Boolean>(storageVolumes.size)
    for (storageVolume in storageVolumes) {
        val volumePath = FileUtilEx.getVolumePath(storageVolume)
        val hasAccess = volumePath != null && storageVolumePathsWeHaveAccessTo.contains(volumePath)
        result[storageVolume] = hasAccess
    }
    return result
}