Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Making multiple API calls in a functional way

What would it be the best approach to solve this problem in the most functional (algebraic) way by using Scala and Cats (or maybe another library focused on Category Theory and/or functional programming)?

Resources

Provided we have the following methods which perform REST API calls to retrieve single pieces of information?

type FutureApiCallResult[A] = Future[Either[String, Option[A]]]

def getNameApiCall(id: Int): FutureApiCallResult[String]
def getAgeApiCall(id: Int): FutureApiCallResult[Int]
def getEmailApiCall(id: Int): FutureApiCallResult[String]

As you can see they produce asynchronous results. The Either monad is used to return possible errors during API calls and Option is used to return None whenever the resource is not found by the API (this case is not an error but a possible and desired result type).

Method to implement in a functional way

case class Person(name: String, age: Int, email: String)

def getPerson(id: Int): Future[Option[Person]] = ???

This method should used the three API calls methods defined above to asynchronously compose and return a Person or None if either any of the API calls failed or any of the API calls return None (the whole Person entity cannot be composed)

Requirements

For performance reasons all the API calls must be done in a parallel way

My guess

I think the best option would be to use the Cats Semigroupal Validated but I get lost when trying to deal with Future and so many nested Monads :S

Can anyone tell me how would you implement this (even if changing method signature or main concept) or point me to the right resources? Im quite new to Cats and Algebra in coding but I would like to learn how to handle this kind of situations so that I can use it at work.

like image 458
Enrique Molina Avatar asked Jan 07 '18 14:01

Enrique Molina


People also ask

Can you make multiple API calls at once?

If you need to make multiple API requests, you can send these API requests concurrently instead of sending them one by one. Sometimes, we need to make multiple API calls at once. For example, let's say we have an array, and we want to make an API request for each element of that array.

How do I make a concurrent API call?

Code for concurrent API call: query. tags); // make concurrent api calls const requests = tags. map((tag) => axios. get("https://api.hatchways.io/assessment/blog/posts?tag=" + tag) ); try { // wait until all the api calls resolves const result = await Promise.

How many calls can an API handle?

In the API Console, there is a similar quota referred to as Requests per 100 seconds per user. By default, it is set to 100 requests per 100 seconds per user and can be adjusted to a maximum value of 1,000. But the number of requests to the API is restricted to a maximum of 10 requests per second per user.


2 Answers

The key requirement here is that it has to be done in parallel. It means that the obvious solution using a monad is out, because monadic bind is blocking (it needs the result in case it has to branch on it). So the best option is to use applicative.

I'm not a Scala programmer, so I can't show you the code, but the idea is that an applicative functor can lift functions of multiple arguments (a regular functor lifts functions of single argument using map). Here, you would use something like map3 to lift the three-argument constructor of Person to work on three FutureResults. A search for "applicative future in Scala" returns a few hits. There are also applicative instances for Either and Option and, unlike monads, applicatives can be composed together easily. Hope this helps.

like image 182
Bartosz Milewski Avatar answered Oct 19 '22 11:10

Bartosz Milewski


You can make use of the cats.Parallel type class. This enables some really neat combinators with EitherT which when run in parallel will accumulate errors. So the easiest and most concise solution would be this:

type FutureResult[A] = EitherT[Future, NonEmptyList[String], Option[A]]

def getPerson(id: Int): FutureResult[Person] = 
  (getNameApiCall(id), getAgeApiCall(id), getEmailApiCall(id))
    .parMapN((name, age, email) => (name, age, email).mapN(Person))

For more information on Parallel visit the cats documentation.

Edit: Here's another way without the inner Option:

type FutureResult[A] = EitherT[Future, NonEmptyList[String], A]

def getPerson(id: Int): FutureResult[Person] = 
  (getNameApiCall(id), getAgeApiCall(id), getEmailApiCall(id))
    .parMapN(Person)
like image 45
Luka Jacobowitz Avatar answered Oct 19 '22 12:10

Luka Jacobowitz