How to complete a subscriber when a for loop is done?

Question

If you make a GET request to https://api.github.com/users/benawad/repos you'll get all the repositores that the user benawad has. At least that's what that API endpoint is expected to return. The problem is that GitHub limits the number of repositores to 30.

As of today (14/08/2021), the user benawad has 246 repositores.

To overcome the above mentioned problem you should make a GET request but with some additional params... GitHub has implemented pagination to it's API. So, in the URL you should specify the page you want to retrieve and the amount of repositores per page.

Our new GET request should look something like this:

https://api.github.com/users/benawad/repos?page=1&per_page=1000

The problem with this, is that GitHub has limited the amount of repositores per page to a 100. So our GET request returns a 100 repos and not the 246 that the user benawad currrently has.

In my Angular service I have implemented the following code to retrieve all repos from all the pages.

  public getUserRepos(user: string): Observable<RepositoryI[]> {
    return new Observable((subscriber: Subscriber<RepositoryI[]>) => {
      this.getUserData(user).subscribe((data: UserI) => {
        //public_repos is the amt of repos the user has
        const pages: number = Math.ceil(data.public_repos / 100);

        for (let i = 1; i <= pages; i++) {
          this.http
            .get(
              `https://api.github.com/users/${user}/repos?page=${i}&per_page=100`
            )
            .subscribe((data: RepositoryI[]) => {
              subscriber.next(data);
            });
        }
      });
    });
  }

And in my component I subscribe with the following:

  this.userService.getUserRepos(id).subscribe((repos)=>{
    this.repositories.push(...repos);
  })

The problem with this aproach is that I have no control of when the Observable has stopped emitting values. In my component I would like to trigger a function when the Observable is complete.

I've tried the following:

  public getUserRepos(user: string): Observable<RepositoryI[]> {
    return new Observable((subscriber: Subscriber<RepositoryI[]>) => {
      this.getUserData(user).subscribe((data: UserI) => {
        const pages: number = Math.ceil(data.public_repos / 100);

        for (let i = 1; i <= pages; i++) {
          this.http
            .get(
              `https://api.github.com/users/${user}/repos?page=${i}&per_page=100`
            )
            .subscribe((data: RepositoryI[]) => {
              subscriber.next(data);
              // If the for loop is complete -> complete the subscriber
              if(pages == i) subscriber.complete();
            });
        }
      });
    });
  }

And in my component I do the following:

  this.userService.getUserRepos(id).subscribe(
    (repos) => {
      this.repositories.push(...repos);
    },
    (err) => {
      console.log(err);
    },
    () => {
      // When the observable is complete:
      
      console.log(this.repositories); // This only logs 46 repositores, it should log 246
      // I want to trigger some function here
    }
  );

The console.log() only logs 46 repositores. Why is this happening? Maybe im completing the subscriber before it can fetch all 3 pages; but I'm calling the .complete() inside the subscription. What am I doing wrong? Thanks in advance.

Answer 1

You can achieve that by one subscription using RxJS operators and functions, like the following:

public getUserRepos(user: string): Observable<RepositoryI[]> {
  // will return one observable that contains all the repos after concating them to each other:
  return this.getUserData(user).pipe(
    switchMap((data: UserI) => {
      //public_repos is the amt of repos the user has
      const pages: number = Math.ceil(data.public_repos / 100);

      // forkJoin will emit the result as (RepositoryI[][]) once all the sub-observables are completed:
      return forkJoin(
        Array.from(new Array(pages)).map((_, page) =>
          this.http.get<RepositoryI[]>(
            `https://api.github.com/users/${user}/repos?page=${page + 1}&per_page=100`
          )
        )
      ).pipe(
        // will reduce the RepositoryI[][] to be RepositoryI[] after concating them to each other:
        map((res) =>
          res.reduce((acc, value) => {
            return acc.concat(value);
          }, [])
        )
      );
    })
  );
}

Then in your component, you can subscribe to the observable that will return all the repos after fetching all of them:

  this.userService.getUserRepos(id).subscribe((repos) => {
    this.repositories = repos;
    // You can trigger the function that you need here...
  });