Get filepath from google drive in Lollipop (MediaStore.MediaColumns.DATA == null)

Question

When the user clicks the "send file" button in google drive and selects my app. I want to get the filepath of that file and then allow the user to upload it to a different location.

I check these similar SO post for kitkat phones: Get real path from URI, Android KitKat new storage access framework

Android - Convert URI to file path on lollipop

However the solution to that no longer seems to work in Lollipop devices.

The problem seems to be that MediaStore.MediaColumns.DATA returns null when running a query on the ContentResolver.

https://code.google.com/p/android/issues/detail?id=63651

You should use ContentResolver.openFileDescriptor() instead of trying to get a raw filesystem path. The "_data" column is not part of the CATEGORY_OPENABLE contract, so Drive is not required to return it.

I've read this blog post by CommonsWare which suggest I "try using the Uri directly with ContentResolver" which I don't understand. How do I use the URI directly with ContentResolvers?

However, I'm still not clear on how best to approach these types of URIs.

The best solution i've been able to find is to call openFileDescriptor and then copy the filestream into a new file, then passing that new file path to my upload activity.

 private static String getDriveFileAbsolutePath(Activity context, Uri uri) {
    if (uri == null) return null;
    ContentResolver resolver = context.getContentResolver();
    FileInputStream input = null;
    FileOutputStream output = null;
    String outputFilePath = new File(context.getCacheDir(), fileName).getAbsolutePath();
    try {
        ParcelFileDescriptor pfd = resolver.openFileDescriptor(uri, "r");
        FileDescriptor fd = pfd.getFileDescriptor();
        input = new FileInputStream(fd);
        output = new FileOutputStream(outputFilePath);
        int read = 0;
        byte[] bytes = new byte[4096];
        while ((read = input.read(bytes)) != -1) {
            output.write(bytes, 0, read);
        }
        return new File(outputFilePath).getAbsolutePath();
    } catch (IOException ignored) {
        // nothing we can do
    } finally {
            input.close();
            output.close();
    }
    return "";
}

The only problem here is that I lose the filename of that file. This seems a bit over complicated just to get a filePath from drive. Is there a better way to do this?

thanks.

EDIT: So i can use a normal query to get the filename. Then I can pass that into my getDriveAbsolutePath() method. Which will get me pretty close to what I want, the only problem now is that I'm missing file extensions. All searches I've done recommend using the filepath to get extensions, which I can't do with openFileDescriptor(). Any help?

    String filename = "";
    final String[] projection = {
            MediaStore.MediaColumns.DISPLAY_NAME
    };
    ContentResolver cr = context.getApplicationContext().getContentResolver();
    Cursor metaCursor = cr.query(uri, projection, null, null, null);
    if (metaCursor != null) {
        try {
            if (metaCursor.moveToFirst()) {
                filename = metaCursor.getString(0);
            }
        } finally {
            metaCursor.close();
        }
    }

However, I'm not entirely convinced this is the "right" way to do this?

Answer 1

The only problem here is that I lose the filename of that file. This seems a bit over complicated just to get a filePath from drive. Is there a better way to do this?

You seem to miss an important point here. Files in Linux don't need to have a name. They may exist in memory (e.g. android.os.MemoryFile) or even reside in directory without having a name (such as files, created with O_TMPFILE flag). What they do need to have is a file descriptor.

---

Short summary: file descriptors are better than simple files and should always be used instead, unless closing them after yourself is too much of burden. They can be employed for same things as File objects, and much more, if you can use JNI. They are made available by special ContentProvider and can be accessed via openFileDescriptor method of ContentResolver (which receives Uri, associated with target provider).

That said, just saying people, used to File objects, to replace them with descriptors sure sounds weird. Read an elaborate explanation below, if you feel like trying it out. If you don't, just skip to the bottom of the answer for "simple" solution.

EDIT: the answer below have been written before Lollipop became widespread. Nowadays there is a handy class for direct access to Linux system calls, that makes using JNI for working with file descriptors optional.

---

Quick briefing on descriptors

File descriptors come from Linux open system call and corresponding open() function in C library. You don't need to have access to file to operate on it's descriptor. Most access checks will simply be skipped, but some crucial information, such as access type (read/write/read-and-write etc.) is "hardcoded" into descriptor and can not be changed after it is created. File descriptors are represented by non-negative integer numbers, starting from 0. Those numbers are local to each process and don't have any persistent or system-wide meaning, they merely distinguish handles to files from each other for given process (0, 1 and 2 traditionally reference stdin, stdout and stderr).

Each descriptor is represented by a reference to entry in descriptor table, stored in OS kernel. There are per-process and system-wide limits to number of entries in that table, so close your descriptors quickly, unless you want your attempts to open things and create new descriptors to suddenly fail.

Operating on descriptors

In Linux there are two kinds of C library functions and system calls: working with names (such as readdir(), stat(), chdir(), chown(), open(), link()) and operating on descriptors: getdents, fstat(), fchdir(), fchown(), fchownat(), openat(), linkat() etc. You can call these functions and system calls easily after a reading a couple of man pages and studying some dark JNI magic. That will raise quality of your software through the roof! (just in case: I am talking about reading and studying, not just blindly using JNI all the time).

In Java there is a class for working with descriptors: java.io.FileDescriptor. It can be used with FileXXXStream classes and thus indirectly with all framework IO classes, including memory-mapped and random access files, channels and channel locks. It is a tricky class. Because of requirement to be compatible with certain proprietary OS, this cross-platform class does not expose underlying integer number. It can not even be closed! Instead you are expected to close the corresponding IO classes, which (again, for compatibility reasons) share the same underlying descriptor with each other:

FileInputStream fileStream1 = new FileInputStream("notes.db");
FileInputStream fileStream2 = new FileInputStream(fileStream1.getFD());
WritableByteChannel aChannel = fileStream1.getChannel();

// pass fileStream1 and aChannel to some methods, written by clueless people
...

// surprise them (or get surprised by them)
fileStream2.close();

There are no supported ways to get integer value out of FileDescriptor, but you can (almost) safely assume, that on older OS versions there is a private integer descriptor field, which can be accessed via reflection.

Shooting yourself in the foot with descriptors

In Android framework there is a specialized class for working with Linux file descriptor: android.os.ParcelFileDescriptor. Unfortunately, it is almost as bad as FileDescriptor. Why? For two reasons:

1) It has a finalize() method. Read it's javadoc to learn, what this means for your performance. And you still has to close it, if you don't want to face sudden IO errors.

2) Because of being finalizable it will be automatically closed by virtual machine once the reference to a class instance goes out of scope. Here is why having finalize() on some framework classes, especially MemoryFile is a mistake on part of framework developers:

public FileOutputStream giveMeAStream() {
  ParcelFileDescriptor fd = ParcelFileDescriptor.open("myfile", MODE_READ_ONLY);

  return new FileInputStream(fd.getDescriptor());
}

...

FileInputStream aStream = giveMeAStream();

// enjoy having aStream suddenly closed during garbage collection

Fortunately, there is a remedy to such horrors: a magical dup system call:

public FileOutputStream giveMeAStream() {
  ParcelFileDescriptor fd = ParcelFileDescriptor.open("myfile", MODE_READ_ONLY);

  return new FileInputStream(fd.dup().getDescriptor());
}

...

FileInputStream aStream = giveMeAStream();

// you are perfectly safe now...




// Just kidding! Also close original ParcelFileDescriptor like this:
public FileOutputStream giveMeAStreamProperly() {
  // Use try-with-resources block, because closing things in Java is hard.
  // You can employ Retrolambda for backward compatibility,
  // it can handle those too!      
  try (ParcelFileDescriptor fd = ParcelFileDescriptor.open("myfile", MODE_READ_ONLY)) {

    return new FileInputStream(fd.dup().getDescriptor());
  }
}

The dup syscall clones integer file descriptor, which makes corresponding FileDescriptor independent from original one. Note, that passing descriptors across processes does not require manual duplication: received descriptors are independent from source process. Passing descriptor of MemoryFile (if you obtain it with reflection) does require the call to dup: having a shared memory region destroyed in originating process will make it inaccessible to everyone. Furthermore, you have to either perform dup in native code or keep a reference to created ParcelFileDescriptor until a receiver is done with your MemoryFile.

Giving and receiving descriptors

There are two ways to give and receive file descriptors: by having a child process inherit creator's descriptors and via interprocess communication.

Letting children of process inherit files, pipes and sockets, open by creator, is a common practice in Linux, but requires forking in native code on Android – Runtime.exec() and ProcessBuilder close all extra descriptors after creating a child process. Make sure to close unnecessary descriptors too, if you choose to fork yourself.

The only IPC facilities, currently supporting passing file descriptors on Android are Binder and Linux domain sockets.

Binder allows you to give ParcelFileDescriptor to anything that accepts parcelable objects, including putting them in Bundles, returning from content providers and passing via AIDL calls to services.

Note, that most attempts to pass Bundles with descriptors outside of the process, including calling startActivityForResult will by denied by system, likely because timely closing those descriptors would have been too hard. Much better choices are creating a ContentProvider (which manages descriptor lifecycle for you, and publishes files via ContentResolver) or writing an AIDL interface and closing a descriptor right after it is transferred. Also note, that persisting ParcelFileDescriptor anywhere does not make much sense: it would only work until process death and corresponding integer will most likely point to something else, once your process is recreated.

Domain sockets are low-level and a bit painful to use for descriptor transfer, especially compared to providers and AIDL. They are, however, a good (and the only documented) option for native processes. If you are forced to open files and/or move data around with native binaries (which is usually the case for applications, using root privileges), consider not wasting your efforts and CPU resource on intricate communications with those binaries, instead write an open helper. [shameless advert] By the way, you may use the one I wrote, instead of creating your own. [/shameless advert]

---

Answer to exact question

I hope, that this answer have given you a good idea, what's wrong with MediaStore.MediaColumns.DATA, and why creating this column is a misnomer on the part of Android development team.

That said, if you are still not convinced, want that filename at all costs, or simply failed to read the overwhelming wall of text above, here – have a ready-to-go JNI function; inspired by Getting Filename from file descriptor in C (EDIT: now has a pure-Java version):

// src/main/jni/fdutil.c
JNIEXPORT jstring Java_com_example_FdUtil_getFdPathInternal(JNIEnv *env, jint descriptor)
{
  // The filesystem name may not fit in PATH_MAX, but all workarounds
  // (as well as resulting strings) are prone to OutOfMemoryError.
  // The proper solution would, probably, include writing a specialized   
  // CharSequence. Too much pain, too little gain.
  char buf[PATH_MAX + 1] = { 0 };

  char procFile[25];

  sprintf(procFile, "/proc/self/fd/%d", descriptor);

  if (readlink(procFile, buf, sizeof(buf)) == -1) {
    // the descriptor is no more, became inaccessible etc.
    jclass exClass = (*env) -> FindClass(env, "java/io/IOException");

    (*env) -> ThrowNew(env, exClass, "readlink() failed");

    return NULL;
  }

  if (buf[PATH_MAX] != 0) {
    // the name is over PATH_MAX bytes long, the caller is at fault
    // for dealing with such tricky descriptors
    jclass exClass = (*env) -> FindClass(env, "java/io/IOException");

    (*env) -> ThrowNew(env, exClass, "The path is too long");

    return NULL;
  }

  if (buf[0] != '/') {
    // the name is not in filesystem namespace, e.g. a socket,
    // pipe or something like that
    jclass exClass = (*env) -> FindClass(env, "java/io/IOException");

    (*env) -> ThrowNew(env, exClass, "The descriptor does not belong to file with name");

    return NULL;
  }

  // doing stat on file does not give any guarantees, that it
  // will remain valid, and on Android it likely to be
  // inaccessible to us anyway let's just hope
  return (*env) -> NewStringUTF(env, buf);
}

And here is a class, that goes with it:

// com/example/FdUtil.java
public class FdUtil {
  static {
    System.loadLibrary(System.mapLibraryName("fdutil"));
  }

  public static String getFdPath(ParcelFileDescriptor fd) throws IOException {
    int intFd = fd.getFd();

    if (intFd <= 0)
      throw new IOException("Invalid fd");

      return getFdPathInternal(intFd);
  }

  private static native String getFdPathInternal(int fd) throws IOException;
}