PDF file download using BlockingQueue

Question

I'm trying to download a pdf file using URLConnection. Here's how I setup the connection object.

URL serverUrl = new URL(url);
urlConnection = (HttpURLConnection) serverUrl.openConnection();
urlConnection.setDoInput(true);
urlConnection.setRequestMethod("GET");
urlConnection.setRequestProperty("Content-Type", "application/pdf");
urlConnection.setRequestProperty("ENCTYPE", "multipart/form-data");
String contentLength = urlConnection.getHeaderField("Content-Length");

I obtained inputstream from the connection object.

bufferedInputStream = new BufferedInputStream(urlConnection.getInputStream());

And the output stream to write the file contents.

File dir = new File(context.getFilesDir(), mFolder);
if(!dir.exists()) dir.mkdir();
final File f = new File(dir, String.valueOf(documentName));
f.createNewFile();
final BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(new FileOutputStream(f, true)); //true for appendMode

BlockingQueue is created so that threads performing read and write operations can access the queue.

final BlockingQueue<ByteArrayWrapper> blockingQueue = new ArrayBlockingQueue<ByteArrayWrapper>(MAX_VALUE,true);
final byte[] dataBuffer = new byte[MAX_VALUE];

Now created thread to read data from InputStream.

Thread readerThread = new Thread(new Runnable() {
       @Override
       public void run() {
         try {
            int count = 0;
            while((count = bufferedInputStream.read(dataBuffer, 0, dataBuffer.length)) != -1) {
                 ByteArrayWrapper byteArrayWrapper = new ByteArrayWrapper(dataBuffer);
                 byteArrayWrapper.setBytesReadCount(count);
                 blockingQueue.put(byteArrayWrapper);
             }
             blockingQueue.put(null); //end of file
          } catch(Exception e) {
                 e.printStackTrace();
          } finally {
              try {
                 bufferedInputStream.close();
              } catch (IOException e) {
                  e.printStackTrace();
              }
          }
       }
 });

Now the writer thread reads those file contents.

Thread writerThread = new Thread(new Runnable() {
       @Override
       public void run() {
         try {
            while(true) {
               ByteArrayWrapper byteWrapper = blockingQueue.take();
               if(null == byteWrapper) break;
               bufferedOutputStream.write(byteWrapper.getBytesRead(), 0, byteWrapper.getBytesReadCount());
             }
             bufferedOutputStream.flush();
         } catch(Exception e) {
              e.printStackTrace();
         } finally {
              try {
                 bufferedOutputStream.close();
              } catch (IOException e) {
                  e.printStackTrace();
              }
         }
      }
});

Finally, threads are started.

readerThread.start();
writerThread.start();

Theoretically it should read the file from InputStream and save it to the target file. However, in reality, it produces blank pdf file. At some other time, it shows invalid pdf format exception. File size matches with content length of the InputStream. Is there anything I'm missing?

Answer 1

I'm not familiar with ByteArrayWrapper. Does it just hold a reference to the array, like this?

public class ByteArrayBuffer {
    final private byte[] data;

    public ByteArrayBuffer(byte[] data) {
        this.data = data;
    }

    public byte[] getBytesRead() {
        return data;
    }

    /*...etc...*/
} 

If so. that would be the problem: all of the ByteArrayWrapper objects are backed by the same array. Which is repeatedly overwritten by the writer. Even though BlockingQueue did the hard work of safely publishing each object from one thread to the other.

The simplest fix might be to make the ByteArrayWrapper effectively immutable i.e. don't change it after publishing it to another thread. Taking a copy of the array on construction would be simplest:

public ByteArrayWrapper(byte[] data) {
    this.data = Arrays.copyOf(data, data.length);
}

One other problem is that "BlockingQueue does not accept null elements" (see BlockingQueue docs), and so the "end of input" sentinel value doesn't work. Replacing null with a

private static ByteArrayWrapper END = new ByteArrayWrapper(new byte[]{});

in the appropriate places will fix that.

By making those changes to a copy of the code I was able to retrieve a faithful copy of a PDF file.