Aggregate results from multiple HTTP requests to a single list

Question

I am parsing several Json links, and trying to add all output to one List. However the list always gets overwritten instead to only include the results from one of the links:

public class GetShopifyJsonData extends GetRawData {
    private String LOG_TAG = GetShopifyJsonData.class.getSimpleName();
    private List<Product> mProduct;
    private Uri mDestination;

    public GetShopifyJsonData(int page) {
        super(null);
        createUri(page);
        mProduct = new ArrayList<Product>();
    }

    public void execute(){
        super.setRawUrl(mDestination.toString());
        DownloadShopifyData downloadShopifyData = new DownloadShopifyData();
        Log.v(LOG_TAG, "Built URI = " + mDestination.toString());
        downloadShopifyData.execute(mDestination.toString());
    }

    public boolean createUri(int page) {
        final String SHOPIFY_BASE_URL = "";
        final String SHOPIFY_PAGE_PARAM = "page";

        mDestination = Uri.parse(SHOPIFY_BASE_URL).buildUpon()
                .appendQueryParameter(SHOPIFY_PAGE_PARAM, String.valueOf(page)).build();

        return mDestination != null;
    }

    public void processResults() {

        if(getDownloadStatus() != DownloadStatus.OK){
            Log.e(LOG_TAG, "Error Downloading Raw Data");
            return;
        }

        final String SH_PRODUCTS = "products";
        final String SH_TYPE = "product_type";
        final String SH_VARIANTS = "variants";
        final String SH_TITLE = "title";
        final String SH_PRICE = "price";
        final String SH_GRAMS = "grams";

        try {
            JSONObject jsonData = new JSONObject(getData());
            JSONArray productsArray = jsonData.getJSONArray(SH_PRODUCTS);
            for (int i=0; i<productsArray.length(); i++ ) {
                JSONObject jsonProduct = productsArray.getJSONObject(i);
                String productType =jsonProduct.getString(SH_TYPE);
                String title = jsonProduct.getString(SH_TITLE);

                JSONArray variantsArray = jsonProduct.getJSONArray(SH_VARIANTS);
                JSONObject variantProduct = variantsArray.getJSONObject(0);
                String variantTitle = variantProduct.getString(SH_TITLE);
                double price = variantProduct.getDouble(SH_PRICE);
                int grams = variantProduct.getInt(SH_GRAMS);

                if (productType.equals("Keyboard") || productType.equals("Computer")) {
                    Product productObject = new Product(title, price, grams, productType, variantTitle);
                    this.mProduct.add(productObject);
                }
            }

            for(Product singleProduct : mProduct){
                Log.v(LOG_TAG, singleProduct.toString());
                Log.v(LOG_TAG, String.valueOf(mProduct.size()));
            }
        } catch (JSONException jsone) {

            jsone.printStackTrace();
            Log.e(LOG_TAG, "Error Processing JSON data");
        }
    }
}

And the call from MainActivity:

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    for (int i=1; i<6; i++) {
        GetShopifyJsonData jsonData = new GetShopifyJsonData(i);
        jsonData.execute();
    }
}

What do I need to change to get the products to add to each other in a single list?

Answer 1

You are overwriting jsonData each time you iterate through this loop, without storing the results of each previous loop elsewhere.

for (int i=1; i<6; i++) {
    GetShopifyJsonData jsonData = new GetShopifyJsonData(i); // throws out results of previous iteration and creates a new List each time
    jsonData.execute();
}

You should keep a List outside of this loop that you can add all of your results to after each iteration:

ArrayList<Product> allProducts = new ArrayList<Product>();
for (int i=1; i<6; i++) {
    GetShopifyJsonData jsonData = new GetShopifyJsonData(i);
    jsonData.execute();
    allProducts.addAll(jsonData.getProducts()) // add a method that gets all of the products from each iteration
}

---

EDIT:

Since you are using threads to collect the JSON data, you can use a List of GetShopifyJsonData objects to maintain references to those threads;

ArrayList<GetShopifyJsonData> allJSONData = new ArrayList<GetShopifyJsonData>();
for (int i=1; i<6; i++) {
    allJSONData.add(new GetShopifyJsonData(i));
    allJSONData.get(i).execute(); // executes each task as discrete instances
}

From there you can check the status of the threads, and retrieve the JSON data from the list as they complete. Below is a not-very-good example, for illustration's sake:

ArrayList<Product> allProducts = new ArrayList<Product>();
for (int i=1; i<6; i++) {
    while(!allJSONData.get(i).isComplete()){ // add a method that checks if a task has been completed
        //this is a busy wait, don't do this!
    }
    allProducts.addAll(jsonData.get(i).getProducts()) // add a method that gets all of the products from each iteration
}

Now, I'm not an android expert, but a quick skim over the docs for AsyncTask shows me that onPostExecute(Result) and getStatus() would probably be useful. If I'm correct, I expect you could actually add the JSON data to a List inside of onPostExecute() and skip using an ArrayList<GetShopifyJsonData> altogether by passing the master list into execute(); something along the lines of:

ArrayList<Product> allProducts = new ArrayList<Product>();
for (int i=1; i<6; i++) {
    GetShopifyJsonData jsonData = new GetShopifyJsonData(i);
    jsonData.execute(allProducts); // pass the reference to your list in execute()
}

// in your AsyncTask class:
private ArrayList<Product> products; // private List instance

execute(ArrayList<Product> allProducts){
    products = allProducts;
    // do other logic
    ...
}

onPostExecute(List<Product> dataFromJSON){
    products.addAll(dataFromJSON); // add the results to the instance variable, which also updates the master list
}

This is an oversimplification, though. You would have to make sure that threads don't try to add to the List at the same time, because bad things can potentially happen if they are allowed to do so. The documentation says that onPostExecute() runs on the UI thread, but I don't know what that means.

Answer 2

Each GetShopifyJsonData has its own List<Product> and you need to aggregate those by either having a single GetShopifyJsonData instance request all the products or by having your MainActivity aggregate them as the requests complete. This approach implements the latter.

Add a callback interface to GetShopifyJsonData and require an instance of it as a parameter in the constructor. I only included the changes in the code below. Everything else is the same.

public class GetShopifyJsonData extends GetRawData {
    public interface OnResultsReadyListener {
        void onResultsReady(List<Product> products);
    }

    private OnResultsReadyListener mResultsListener;

    public GetShopifyJsonData(int page, OnResultsReadyListener resultsListener) {
        super(null);
        createUri(page);
        mProduct = new ArrayList<Product>();
        mResultsListener = resultsListener;
    }

    public void processResults() {
        // Add this to the end of the method
        if(mResultsListener != null) {
            mResultsListener.onResultsReady(mProduct);
        }
    }
}

And then update MainActivity to implement this new interface and add results to its list as requests complete.

public class MainActivity extends Activity
        implements GetShopifyJsonData.OnResultsReadyListener {
    private List<Product> allproducts;

    @Override
    void onResultsReady(List<Product> products) {
        // allProducts contains products for all requests that have completed so far
        allProducts.addAll(products);
        Log.v(LOG_TAG, allProducts.size() + " total products downloaded.");
    }

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        allProducts = new ArrayList<>();

        for (int i=1; i<6; i++) {
            GetShopifyJsonData jsonData = new GetShopifyJsonData(i, this);
            jsonData.execute();
        }
    }
}