Shortest code to cache Rxjs http request while not complete?

Question

I'm trying to create an observable flow that fulfills the following requirements:

1. Loads data from storage at subscribe time
2. If the data has not yet expired, return an observable of the stored value
3. If the data has expired, return an HTTP request observable that uses the refresh token to get a new value and store it
   • If this code is reached again before the request has completed, return the same request observable
   • If this code is reached after the previous request completed or with a different refresh token, start a new request

I'm aware that there are many different answers on how to perform step (3), but as I'm trying to perform these steps together I am looking for guidance on whether the solution I've come up with is the most succinct it can be (which I doubt!).

Here's a sample demonstrating my current approach:

var cachedRequestToken;
var cachedRequest;

function getOrUpdateValue() {
    return loadFromStorage()
      .flatMap(data => {
        // data doesn't exist, shortcut out
        if (!data || !data.refreshtoken) 
            return Rx.Observable.empty();

        // data still valid, return the existing value
        if (data.expires > new Date().getTime())
            return Rx.Observable.return(data.value);

        // if the refresh token is different or the previous request is 
        // complete, start a new request, otherwise return the cached request
        if (!cachedRequest || cachedRequestToken !== data.refreshtoken) { 
            cachedRequestToken = data.refreshtoken;

            var pretendHttpBody = {
                value: Math.random(),
                refreshToken: Math.random(),
                expires: new Date().getTime() + (10 * 60 * 1000) // set by server, expires in ten minutes
            };

            cachedRequest = Rx.Observable.create(ob => { 
                // this would really be a http request that exchanges
                // the one use refreshtoken for new data, then saves it
                // to storage for later use before passing on the value

                window.setTimeout(() => { // emulate slow response   
                    saveToStorage(pretendHttpBody);
                    ob.next(pretendHttpBody.value);                 
                    ob.completed(); 
                    cachedRequest = null; // clear the request now we're complete
                }, 2500);
          });
        }

        return cachedRequest;
    });
}

function loadFromStorage() {
    return Rx.Observable.create(ob => {
        var storedData = {  // loading from storage goes here
            value: 15,      // wrapped in observable to delay loading until subscribed
            refreshtoken: 63, // other process may have updated this between requests
            expires: new Date().getTime() - (60 * 1000) // pretend to have already expired
        };

        ob.next(storedData);
        ob.completed();
    })
}

function saveToStorage(data) {
    // save goes here
}

// first request
getOrUpdateValue().subscribe(function(v) { console.log('sub1: ' + v); }); 

// second request, can occur before or after first request finishes
window.setTimeout(
    () => getOrUpdateValue().subscribe(function(v) { console.log('sub2: ' + v); }),
    1500); 

Answer 1

First, have a look at a working jsbin example.

The solution is a tad different then your initial code, and I'd like to explain why. The need to keep returning to your local storage, save it, save flags (cache and token) didn't not fit for me with reactive, functional approach. The heart of the solution I gave is:

var data$ = new Rx.BehaviorSubject(storageMock);
var request$ = new Rx.Subject();
request$.flatMapFirst(loadFromServer).share().startWith(storageMock).subscribe(data$);
data$.subscribe(saveToStorage);

function getOrUpdateValue() {
    return data$.take(1)
      .filter(data => (data && data.refreshtoken))
      .switchMap(data => (data.expires > new Date().getTime() 
                    ? data$.take(1)
                    : (console.log('expired ...'), request$.onNext(true) ,data$.skip(1).take(1))));
}

The key is that data$ holds your latest data and is always up to date, it is easily accessible by doing a data$.take(1). The take(1) is important to make sure your subscription gets a single values and terminates (because you attempt to work in a procedural, as opposed to functional, manner). Without the take(1) your subscription would stay active and you would have multiple handlers out there, that is you'll handle future updates as well in a code that was meant only for the current update.

In addition, I hold a request$ subject which is your way to start fetching new data from the server. The function works like so:

1. The filter ensures that if your data is empty or has no token, nothing passes through, similar to the return Rx.Observable.empty() you had.
2. If the data is up to date, it returns data$.take(1) which is a single element sequence you can subscribe to.
3. If not, it needs a refresh. To do so, it triggers request$.onNext(true) and returns data$.skip(1).take(1). The skip(1) is to avoid the current, out dated value.

For brevity I used (console.log('expired ...'), request$.onNext(true) ,data$.skip(1).take(1))). This might look a bit cryptic. It uses the js comma separated syntax which is common in minifiers/uglifiers. It executes all statements and returns the result of the last statement. If you want a more readable code, you could rewrite it like so:

.switchMap(data => {
  if(data.expires > new Date().getTime()){ 
    return data$.take(1);
  } else {
    console.log('expired ...');
    request$.onNext(true);
    return data$.skip(1).take(1);
  }
});

The last part is the usage of flatMapFirst. This ensures that once a request is in progress, all following requests are dropped. You can see it works in the console printout. The 'load from server' is printed several times, yet the actual sequence is invoked only once and you get a single 'loading from server done' printout. This is a more reactive oriented solution to your original refreshtoken flag checking.

Though I didn't need the saved data, it is saved because you mentioned that you might want to read it on future sessions.

A few tips on rxjs:

1. Instead of using the setTimeout, which can cause many problems, you can simply do Rx.Observable.timer(time_out_value).subscribe(...).

2. Creating an observable is cumbersome (you even had to call next(...) and complete()). You have a much cleaner way to do this using Rx.Subject. Note that you have specifications of this class, the BehaviorSubject and ReplaySubject. These classes are worth knowing and can help a lot.

One last note. This was quite a challange :-) I'm not familiar with your server side code and design considerations yet the need to suppress calls felt uncomfortable to me. Unless there is a very good reason related to your backend, my natural approach would be to use flatMap and let the last request 'win', i.e. drop previous un terminated calls and set the value.

The code is rxjs 4 based (so it can run in jsbin), if you're using angular2 (hence rxjs 5), you'll need to adapt it. Have a look at the migration guide.

================ answers to Steve's other questions (in comments below) =======

There is one article I can recommend. It's title says it all :-)

As for the procedural vs. functional approach, I'd add another variable to the service:

let token$ = data$.pluck('refreshtoken');

and then consume it when needed.

My general approach is to first map my data flows and relations and then like a good "keyboard plumber" (like we all are), build the piping. My top level draft for a service would be (skipping the angular2 formalities and provider for brevity):

class UserService {
  data$: <as above>;
  token$: data$.pluck('refreshtoken');
  private request$: <as above>;

  refresh(){
    request.onNext(true);
  }
}

You might need to do some checking so the pluck does not fail.

Then, each component that needs the data or the token can access it directly.

Now lets suppose you have a service that needs to act on a change to the data or the token:

class SomeService {
  constructor(private userSvc: UserService){
    this.userSvc.token$.subscribe(() => this.doMyUpdates());
  }
}

If your need to synthesize data, meaning, use the data/token and some local data:

Rx.Observable.combineLatest(this.userSvc.data$, this.myRelevantData$)
  .subscribe(([data, myData] => this.doMyUpdates(data.someField, myData.someField));

Again, the philosophy is that you build the data flow and pipes, wire them up and then all you have to do is trigger stuff.

The 'mini pattern' I've come up with is to pass to a service once my trigger sequence and register to the result. Lets take for example autocomplete:

class ACService {
   fetch(text: string): Observable<Array<string>> {
     return http.get(text).map(response => response.json().data;
   }
}

Then you have to call it every time your text changes and assign the result to your component:

<div class="suggestions" *ngFor="let suggestion; of suggestions | async;">
  <div>{{suggestion}}</div>
</div>

and in your component:

onTextChange(text) {
  this.suggestions = acSVC.fetch(text);
}

but this could be done like this as well:

class ACService {
   createFetcher(textStream: Observable<string>): Observable<Array<string>> {
     return textStream.flatMap(text => http.get(text))
       .map(response => response.json().data;
   }
}

And then in your component:

textStream: Subject<string> = new Subject<string>();
suggestions: Observable<string>;

constructor(private acSVC: ACService){
  this.suggestions = acSVC.createFetcher(textStream);
}

onTextChange(text) {
  this.textStream.next(text);
}

template code stays the same.

It seems like a small thing here, but once the app grows bigger, and the data flow complicated, this works much better. You have a sequence that holds you data and you can use it around the component wherever you need it, you can even further transform it. For example, lets say you need to know the number of suggestions, in the first method, once you get the result, you need to further query it to get it, thus:

onTextChange(text) {
  this.suggestions = acSVC.fetch(text);
  this.suggestionsCount = suggestions.pluck('length'); // in a sequence
  // or
  this.suggestions.subscribe(suggestions => this.suggestionsCount = suggestions.length); // in a numeric variable.
}

Now in the second method, you just define:

constructor(private acSVC: ACService){
  this.suggestions = acSVC.createFetcher(textStream);
  this.suggestionsCount = this.suggestions.pluck('length');
}

Hope this helps :-)

While writing, I tried to reflect about the path I took to getting to use reactive like this. Needless to say that on going experimentation, numerous jsbins and strange failures are big part of it. Another thing that I think helped shape my approach (though I'm not currently using it) is learning redux and reading/trying a bit of ngrx (angular's redux port). The philosophy and the approach does not let you even think procedural so you have to tune in to functional, data, relations and flows based mindset.