How to publish progress for large json file parsing with GSON

Question

Dear Stackoverflowers,

I am currently parsing a large json file from my raw resources. I had to change reading line by line to using a Reader object in combination with gson, to escape an out of memory exception. So far, so good.

Now this all happens in an async task and I want to have the user be notified of the progress in some kind of loading screen by using publishProgress().

InputStream raw = getResources().openRawResource(R.raw.json);
Reader rd = new BufferedReader(new InputStreamReader(raw));
Gson gson = new Gson();
mReadObjects = gson.fromJson(rd, ReadObjectList.class);

This is the way I'm reading the file for now, but I have no clue if (and how) I can get any kind of progress updates from GSON or the Reader object.

Any help is greatly appreciated!

Answer 1

You have to write a Wrapper around an InputStream (or a Reader).

Something like this should work:

public class ProgressInputStream extends FilterInputStream {
    private final int size;
    private long bytesRead;
    private int percent;
    private List<Listener> listeners = new ArrayList<>();

    public ProgressInputStream(InputStream in) {
        super(in);
        try {
            size = available();
            if (size == 0) throw new IOException();
        } catch (IOException e) {
            throw new RuntimeException("This reader can only be used on InputStreams with a known size", e);
        }
        bytesRead = 0;
        percent = 0;
    }

    public void addListener(Listener listener) {
        listeners.add(listener);
    }

    public void removeListener(Listener listener) {
        listeners.remove(listener);
    }

    @Override
    public int read() throws IOException {
        int b = super.read();
        updateProgress(1);
        return b;
    }

    @Override
    public int read(@NonNull byte[] b) throws IOException {
        return updateProgress(super.read(b));
    }

    @Override
    public int read(@NonNull byte[] b, int off, int len) throws IOException {
        return updateProgress(super.read(b, off, len));
    }

    @Override
    public long skip(long n) throws IOException {
        return updateProgress(super.skip(n));
    }

    @Override
    public void mark(int readLimit) {
        throw new UnsupportedOperationException();
    }

    @Override
    public void reset() throws IOException {
        throw new UnsupportedOperationException();
    }

    @Override
    public boolean markSupported() {
        return false;
    }

    private <T extends Number> T updateProgress(T numBytesRead) {
        if (numBytesRead.longValue() > 0) {
            bytesRead += numBytesRead.longValue();
            if (bytesRead * 100 / size > percent) {
                percent = (int) (bytesRead * 100 / size);
                for (Listener listener : listeners) {
                    listener.onProgressChanged(percent);
                }
            }
        }
        return numBytesRead;
    }

    public interface Listener {
        void onProgressChanged(int percentage);
    }
}

How to use:

ProgressInputStream raw = new ProgressInputStream(getResources().openRawResource(R.raw.json));
raw.addListener(new ProgressInputStream.Listener() {
    @Override
    public void onProgressChanged(int percentage) {
        publishProgress(percentage);
    }
});
Reader rd = new BufferedReader(new InputStreamReader(raw));