Python tips for working with an unstable `API`

Question

My app is using a third-party API. It polls that API regularly at several endpoints. In also makes some additional calls to the API based on user's interaction with the app. The API is very slow, most requests take well over a second. The API is very flaky - timeouts are common, 500 errors are common, session key often randomly expires (even when defined "keep_alive" endpoint is called regularly). There is no option to use another API.

What would be the best practices for dealing with such an API?

How to disable concurrent requests to this API on the requests level. So if one request is waiting for a response - the second request is not initiated? This should be done on "per-domain" basis, other requests to other domains should still be done concurrently.

Any other settings to toggle with requests to make it easier to deal with such an API?

Answer 1

What would be the best practices for dealing with such an API?

In SRE we pretty much always assume that API's can never be trusted, Because of this there are a number of patterns that may help:

• Circuit Breaking
• Caching
• Timeouts
• Retry/Backoff

References:

• https://github.com/App-vNext/Polly/wiki/Circuit-Breaker
• https://learn.microsoft.com/en-us/azure/architecture/patterns/category/resiliency
• http://resilience4j.github.io/resilience4j/
• https://www.envoyproxy.io/