API management service ClientConnectionFailure at forward-request

Question

I have a Azure function calling Service fabric API using HttpClient API exposed in API management service. But the API management service logs shows that the API call failed with the problem ID "ClientConnectionFailure at forward-request".

On the client side I receive System.Net.Sockets.SocketException with error message "The operation was canceled. Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request. The I/O operation has been aborted because of either a thread exit or an application request"

What are the possible reasons for this failure? How to fix it? Unfortunately I couldn't find enough documentation on this?

Answer 1

First you have to understand that in APIM we have client which could be your browser or any other tool, the APIM proxy that forwards the requests to the back-end and the back-end that hosts your application.

The reason behind the client connection failure is that the client drops the connection before receiving the response or the complete response from the APIM gateway (proxy).

The client may drop the connection if

1. The APIM gateway takes long time to respond due to heavy load at the APIM gateway or the back-end the APIM instance points to (proxying) is unable to handle the number of requests forwarded by the APIM gateway (unable to keep up with the apim service load) so the client times out while waiting for a response or sometimes while still sending the request.

2. The client drops the connection for any other reasons (user cancelled the request for example, client app closed, crashed ...etc.)

ClientConnectionFailure at forward-request means that the client dropped the connection while the APIM gateway was sending the request to the back end or waiting for a response from the back end, see this What's the difference between transfer-response and forward-request errors in API management?.

You have to identify what reason is causing this and based on that you need to react by either fixing your client or to scale-up your APIM or your back-end.

Answer 2

I had similar issue. In my case sales force is the client and I have observed this error often. Sales forces closes connection if it does not get a response within 5 seconds of call. This fails the request at APIM level and error "ClientConnectionFailure at forward-request" is observed.

Please check for how long does you'r client wait for the response. If APIM takes longer than that is the same issue.

to cater this you need to either increase wait time at client side or work on improving response time of APIM.