Why don't I have permission to write to app dir on external storage?

Question

The TL;DR question summary: My Android app tries to write to the app's external storage directory on an SD card. It fails with a permissions error. But the same code (method), extracted into a minimal test app, succeeds!

Since our target API level includes KitKat and later (as well as JellyBean), and KitKat restricts apps from writing anywhere on the SD card except the app's designated external storage directory, the app tries to write to that designated directory, /path/to/sdcard/Android/data/com.example.myapp/files. I verify this directory's path by getting a list of directories from Activity.getExternalFilesDirs(null); and finding one that isRemovable(). I.e. we're not hard-coding the path to the SD card, because it varies by manufacturer and device. Here is code that demonstrates the problem:

// Attempt to create a test file in dir.
private void testCreateFile(File dir) {
    Log.d(TAG, ">> Testing dir " + dir.getAbsolutePath());

    if (!checkDir(dir)) { return; }

    // Now actually try to create a file in this dir.
    File f = new File(dir, "foo.txt");
    try {
        boolean result = f.createNewFile();
        Log.d(TAG, String.format("Attempted to create file. No errors. Result: %b. Now exists: %b",
                result, f.exists()));
    } catch (Exception e) {
        Log.e(TAG, "Failed to create file " + f.getAbsolutePath(), e);
    }
}

The checkDir() method is not as relevant, but I'll include it here for completeness. It just makes sure the directory is on removable storage that is mounted, and logs other properties of the directory (exists, writable).

private boolean checkDir(File dir) {
    boolean isRemovable = false;
    // Can't tell whether it's removable storage?
    boolean cantTell = false;
    String storageState = null;

    // Is this the primary external storage directory?
    boolean isPrimary = false;
    try {
        isPrimary = dir.getCanonicalPath()
                .startsWith(Environment.getExternalStorageDirectory().getCanonicalPath());
    } catch (IOException e) {
        isPrimary = dir.getAbsolutePath()
                .startsWith(Environment.getExternalStorageDirectory().getAbsolutePath());
    }

    if (isPrimary) {
        isRemovable = Environment.isExternalStorageRemovable();
        storageState = Environment.getExternalStorageState();
    } else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
        // I actually use a try/catch for IllegalArgumentException here, but 
        // that doesn't affect this example.
        isRemovable = Environment.isExternalStorageRemovable(dir);
        storageState = Environment.getExternalStorageState(dir);
    } else {
        cantTell = true;
    }

    if (cantTell) {
        Log.d(TAG, String.format("  exists: %b  readable: %b  writeable: %b primary: %b  cantTell: %b",
                dir.exists(), dir.canRead(), dir.canWrite(), isPrimary, cantTell));
    } else {
        Log.d(TAG, String.format("  exists: %b  readable: %b  writeable: %b primary: %b  removable: %b  state: %s  cantTell: %b",
                dir.exists(), dir.canRead(), dir.canWrite(), isPrimary, isRemovable, storageState, cantTell));
    }

    return (cantTell || (isRemovable && storageState.equalsIgnoreCase(MEDIA_MOUNTED)));
}

In the test app (running on Android 5.1.1), the following log output shows that the code is working fine:

10-25 19:56:40 D/MainActivity: >> Testing dir /storage/extSdCard/Android/data/com.example.testapp/files
10-25 19:56:40 D/MainActivity:   exists: true  readable: true  writeable: true primary: false  removable: true  state: mounted  cantTell: false
10-25 19:56:40 D/MainActivity: Attempted to create file. No errors. Result: false. Now exists: true

So the file was created successfully. But in my actual app (also running on Android 5.1.1), the call to createNewFile() fails with a permissions error:

10-25 18:14:56... D/LessonsDB: >> Testing dir /storage/extSdCard/Android/data/com.example.myapp/files
10-25 18:14:56... D/LessonsDB:   exists: true  readable: true  writeable: true primary: false  removable: true  state: mounted  cantTell: false
10-25 18:14:56... E/LessonsDB: Failed to create file /storage/extSdCard/Android/data/com.example.myapp/files/foo.txt
    java.io.IOException: open failed: EACCES (Permission denied)
        at java.io.File.createNewFile(File.java:941)
        at com.example.myapp.dmm.LessonsDB.testCreateFile(LessonsDB.java:169)
    ...
     Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
        at libcore.io.Posix.open(Native Method)
        at libcore.io.BlockGuardOs.open(BlockGuardOs.java:186)
        at java.io.File.createNewFile(File.java:934)
    ...

Before you mark this as a duplicate: I have read through several other questions on SO describing permission failures when writing to an SD card under KitKat or later. But none of the causes or solutions given seem to apply to this situation:

• The device is not connected as mass storage. I double-checked. However, MTP is on. (I can't turn it off without unplugging the USB cable that I use for viewing the logs.)
• My manifest includes <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
• I'm targeting API level 22, so I shouldn't have to request permission at run-time (a la Marshmallow). The build.gradle has targetSdkVersion 22 (and buildToolsVersion '21.1.2', compileSdkVersion 23).
• I'm running on KitKat and Lollipop; I don't even have a Marshmallow device. So again, I shouldn't have to request permission at runtime, even if I were targeting API level 23.
• As mentioned above, I'm writing to the designated external storage directory that's supposed to be writeable by my app, even on KitKat.
• The external storage is mounted; the code verifies this.

Summary of when it works and when it doesn't:

• On my pre-KitKat device, both the test app and the real app work fine. They successfully create a file in the app dir on the SD card.
• On my KitKat and Lollipop devices, the test app works fine but the real app doesn't. Instead it shows the error in the above log.

What difference is there between the test app and the real app? Well, obviously the real app has a lot more stuff in it. But I can't see anything that should matter. Both have identical compileSdkVersion, targetSdkVersion, buildToolsVersion, etc. Both are also using compile 'com.android.support:appcompat-v7:23.4.0' as a dependency.

Answer 1

Since one app worked and another didn't, the difference was between the apps, not with the device or card. The directory in question does not require any Android permissions (e.g., WRITE_EXTERNAL_STORAGE). The only reason why you would not be able to write to it would be if the Android system had not set up the filesystem permissions properly.

I may have created that directory myself, and failed to set up permissions somewhere?

I am not certain that you can create that directory yourself from outside the app and have it work. Ideally that would be fine, but I have not tried it and I can see where that might pose problems.

Is there another reason not to trust it?

Given the filesystem shenanigans that are going on, I get very nervous when developers make assumptions regarding the nature of paths, that's all.