android LiveData Observable doesn't return data if calls are made quickly

Question

Working on an app which requires sending multiple API calls to the same endpoint in one go.

Eg - Directory browsing scenarios, need to get the directory structure by sending get calls for all folders in a current folder. The issue is, the response comes separately for all folders in retrofit properly, but LiveData observable gives me only one response for the entire list.

Directory structure :-

test -> temp -> temp1 -> temp2 
                      -> temp3
                      -> temp4

Observable to listen for the callback :-

mViewModel.getServerFilesLiveData().observe(this, browseServerDataResource -> {
      if (browseServerDataResource != null) {
        if (browseServerDataResource.status == APIClientStatus.Status.SUCCESS) {
          if (browseServerDataResource.data != null) {
            Timber.i("Got data for path %s in Observable", browseServerDataResource.data.path);
            if (browseServerDataResource.data.folderList != null
              && browseServerDataResource.data.folderList.size() > 0) {
              for (final String name : browseServerDataResource.data.folderList) {
                final ServerDirectoryPathInfo pathInfo = new ServerDirectoryPathInfo();
                pathInfo.completePath = browseServerDataResource.data.path + "/" + name;
                getFolderDownloadPath(pathInfo.completePath);
              }
            }
            mFolderCountToParse--;
            Timber.d("Folders left to parse %d", mFolderCountToParse);
            if (mFolderCountToParse == 0) {
              showToast("Parsed all folders");
            }
          }
        }
      }
    });

Function to make calls to get the data :-

  private void getFolderDownloadPath(@NonNull final String path) {
    mViewModel.getServerFiles(path);
    mFolderCountToParse++;
  }

Retrofit call to server :-

  public LiveData<Resource<BrowseServerData>> getServerFiles(@NonNull final String additionalUrl) {
    final MutableLiveData<Resource<BrowseServerData>> data = new MutableLiveData<>();
    final String url = mMySharedPreferences.getCurrentUrl()
      + AppConstants.DIRECTORY_END_POINT
      + AppConstants.PATH_END_POINT
      + (TextUtils.isEmpty(additionalUrl) ? "" : additionalUrl);
    Timber.i("Requesting data for - api %s", url);
    mAPI.getServerFiles(url, mMySharedPreferences.getNetworkName())
      .enqueue(new Callback<BrowseServerData>() {
        @Override
        public void onResponse(
          @NonNull Call<BrowseServerData> call, @NonNull Response<BrowseServerData> response
        ) {
          if (response.body() != null && response.isSuccessful()) {
            if (!TextUtils.isEmpty(response.body().path)) {
              Timber.i("Got response for = %s in Retrofit", response.body().path);
            }
            data.setValue(
              new Resource<>(APIClientStatus.Status.SUCCESS, response.body(), null, null));
          } else {
            ErrorMessage errorMessage = null;
            try {
              errorMessage = Utility.getApiError(response, mRetrofit);
            } catch (IOException e) {
              e.printStackTrace();
            }
            if (errorMessage != null) {
              data.setValue(
                new Resource<>(APIClientStatus.Status.ERROR, null, errorMessage.message(), call));
            } else {
              data.setValue(
                new Resource<>(APIClientStatus.Status.ERROR, null, response.message(), call));
            }
          }
        }

        @Override
        public void onFailure(@NonNull Call<BrowseServerData> call, @NonNull Throwable throwable) {
          data.setValue(
            new Resource<>(APIClientStatus.Status.ERROR, null, throwable.getMessage(), throwable,
              call));
        }
      });
    return data;
  }

The data comes as :-

I: Got response for = ./test in Retrofit
I: Got data for path ./test in Observable
I: Got response for = ./test/temp in Retrofit
I: Got data for path ./test/temp in Observable
I: Got response for = ./test/temp/temp1 in Retrofit
I: Got data for path ./test/temp/temp1 in Observable
I: Got response for = ./test/temp/temp1/temp2 in Retrofit
I: Got response for = ./test/temp/temp1/temp4 in Retrofit
I: Got response for = ./test/temp/temp1/temp3 in Retrofit
I: Got data for path ./test/temp/temp1/temp3 in Observable

As you can see, data comes in Observable only for one folder temp3.

When added a random delay in making calls, the data comes properly :-

new Handler().postDelayed(new Runnable() {
                  @Override
                  public void run() {
                    getFolderDownloadPath(pathInfo.completePath);
                  }
                }, new Random().nextInt(10000 - 1000) + 1000);

Now atleast data comes for 2 folders out of 3 :-

I: Got response for = . in Retrofit
I: Got data for path . in Observable
I: Got data for the current directory, don't need it, skipping
I: Got response for = ./test in Retrofit
I: Got data for path ./test in Observable
I: Got response for = ./test/temp in Retrofit
I: Got data for path ./test/temp in Observable
I: Got response for = ./test/temp/temp1 in Retrofit
I: Got data for path ./test/temp/temp1 in Observable
I: Got response for = ./test/temp/temp1/temp3 in Retrofit
I: Got response for = ./test/temp/temp1/temp2 in Retrofit
I: Got data for path ./test/temp/temp1/temp2 in Observable
I: Got response for = ./test/temp/temp1/temp4 in Retrofit
I: Got data for path ./test/temp/temp1/temp4 in Observable

Any ideas why this is happening and a way to fix it?

Update :- Adding the ViewModel constructor which helps in making the call to the server

@Inject
  BrowseHubMediaViewModel(@NonNull Application application, @NonNull APIClient mAPIClient) {
    super(application);
    mGetServerFilesMutable = new MutableLiveData<>();
    mGetServerFilesLiveData =
      Transformations.switchMap(mGetServerFilesMutable, mAPIClient::getServerFiles);
}

Getting Observable from ViewModel

  /**
   * Observer to listen for file listing in server
   *
   * @return LiveData<Resource<BrowseServerData>>
   */
  public LiveData<Resource<BrowseServerData>> getServerFilesLiveData() {
    return mGetServerFilesLiveData;
  }

Answer 1

The hint by @niketshah09 prompted me for the solution. Coming to the problem, as @niketshah09 described, the issue was Transformations.switchMap() was removing the last returned call when multiple callback arrives quickly. The solution was to use MediatorLiveData which will merge all calls and make sure that we get all callbacks. Eg -

final LiveData<Resource<BrowseServerData>> newParsingFolderLiveData = mAPIClient.getServerFiles(completePath);
folderBrowsingMediator.addSource(newParsingFolderLiveData, folderBrowsingMediator::setValue);

Next, we have to observe on MediatorLiveData instead of LiveData. Although the functionality of MediatorLiveData is to make sure we filter and use the correct stream, depending on coding logics, but in this case, we want to get all callbacks, so no filtering is applied on callbacks.

This way I get all callbacks, let me know if someone doesn't understand something.