Android / Java: HttpURLConnection doesn't return headers of redirected file (e.g. on S3)

Question

My code (reproduced below), connects to a url and downloads the file to disk on android. All standard stuff. When I try using this code on a file on S3 accessed via a subdomain on our server mapped to a bucket (e.g. foo.example.com => bucket called foo.example.com), it often fails. Turns out (using the handy curl command..

 "curl -v -L -X GET http://foo.example.com/f/a.txt") 

.. that there's a redirect going on here.

The file download works ok, as HttpURLConnection will follow redirects by default, but the calls that require the header infomation (getContentLength, getHeaderFieldDate("Last-Modified", 0 ) etc) are returns the headers from the 307 redirect, and not the actual file thats downloaded.

Anyone know how to get around this?

Thanks

File local = null;
        try {


            Log.i(TAG, "Downloading file " + source);
            conn = (HttpURLConnection) new URL(source).openConnection();
            fileSize = conn.getContentLength(); // ** THIS IS WRONG ON REDIRECTED FILES
            out = new BufferedOutputStream(new FileOutputStream(destination, false), 8 * 1024); 
            conn.connect();

            stream = new BufferedInputStream(conn.getInputStream(), 8 * 1024);

            byte[] buffer = new byte[MAX_BUFFER_SIZE];

            while (true) {
                int read = stream.read(buffer);

                if (read == -1) {
                    break;
                }
                // writing to buffer
                out.write(buffer, 0, read);
                downloaded += read;

                publishProgress(downloaded, fileSize);

                if (isCancelled()) {
                    return "The user cancelled the download"; 
                }

            }
        } catch (Exception e) {
            String msg = "Failed to download file " + source + ". " + e.getMessage();
            Log.e(TAG, msg );
            return msg;
        } finally {
            if (out != null) {
                try {
                    out.flush();
                    out.close();
                } catch (IOException e) {
                    Log.e(TAG, "Failed to close file " + destination);
                    e.printStackTrace();
                }
            }

            if (stream != null) {
                try {
                    stream.close();
                } catch (IOException e) {
                    Log.e(TAG, "Failed to close file " + destination);
                    e.printStackTrace();
                }

            } else {

                long dateLong = conn.getHeaderFieldDate("Last-Modified", 0 ); // ** THIS IS WRONG ON REDIRECTED FILES

                Date d = new Date(dateLong);
                local.setLastModified(dateLong);
            }

Answer 1

have you tried to set redirects to false and try to manually capture the redirected URL and associated header fields with it?

For example something like this:

URL url = new URL(url); 
HttpURLConnection ucon = (HttpURLConnection) url.openConnection(); 
ucon.setInstanceFollowRedirects(false); 
URL secondURL = new URL(ucon.getHeaderField("Location")); 
URLConnection conn = secondURL.openConnection();

This example captures the redirected URL, but you could easily tweak this to try for any other header field. Does this help?