NoHttpResponseException thrown as runtime exception even though is a checked exception

Question

I'm experiencing a strange issue where a org.apache.http.NoHttpResponseException is thrown as unchecked exception even though is a checked exception since it extends java.io.IOException ... As it can be seen from the following posted stacktrace I'm getting an exception that should be checked at compile time as an unchecked runtime exception.

The stacktrace of the exception that I get is as follows(my classes are in the package : com.example.staticsite) :

org.apache.http.NoHttpResponseException: sqs.eu-west-1.amazonaws.com failed to respond
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
    at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
    at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
    at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
    at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
    at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
    at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:66)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
    at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
    at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:728)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
    at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2419)
    at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1130)
    at com.example.staticsite.aws.SqsReceiverImpl.receiveReceipt(SqsReceiverImpl.java:57)
    at com.example.staticsite.core.processsite.ProcessSiteImpl.runOneTime(ProcessSiteImpl.java:59)
    at com.example.staticsite.core.processsite.ProcessSiteImpl.run(ProcessSiteImpl.java:44)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
    at java.lang.Thread.run(Thread.java:748) 

The method that is throwing the exception inside my code is :

public class SqsReceiverImpl implements SqsReceiver {
    private AmazonSQS client;
    private String queueUrl;

    @Inject
    public SqsReceiverImpl(AmazonSQS client,@Assisted String queueUrl) {        
        this.client = client;
        this.queueUrl = queueUrl;
    }

    public List<String> receiveReceipt() throws SqsReceiverException {
        if(queueUrl == null)
            throw new SqsReceiverException(SqsReceiverException.MESSAGE_NO_QUEURURL);
        ReceiveMessageRequest request = new ReceiveMessageRequest();
        request.setMaxNumberOfMessages(10);
        request.setQueueUrl(queueUrl);
        request.setWaitTimeSeconds(20);
    
        ReceiveMessageResult results = null;
        try {
            results = client.receiveMessage(request);
        }
        catch(OverLimitException oe){
            throw new SqsReceiverException("OverLimitException thrown");
        }
        catch(AmazonServiceException oe){
            throw new SqsReceiverException("AmazonServiceException thrown");
        }
        catch(AmazonClientException oe){
            throw new SqsReceiverException("AmazonClientException thrown");
        } 

The SqsReceiverException is defined as follows :

public class SqsReceiverException extends Exception{

    public SqsReceiverException(String messageNoQueururl) {
        super(messageNoQueururl);
    }
    public static final String MESSAGE_NO_QUEURURL = "Queue url not found. Se the queue url";
}

The pom file dependecies are declared as follows :

  <dependencies>
      <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <version>4.11</version>
       <scope>test</scope>
    </dependency>
    <dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-sqs</artifactId>
    <version>1.10.12</version>
</dependency>
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>1.10.19</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>com.google.inject</groupId>
    <artifactId>guice</artifactId>
    <version>4.0</version>
</dependency>
<dependency>
  <groupId>com.google.inject.extensions</groupId>
  <artifactId>guice-assistedinject</artifactId>
  <version>4.0</version>
</dependency>
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-lang3</artifactId>
    <version>3.4</version>
</dependency>
</dependencies>

Producing this results :

How can be possible that this exception is threated as unchecked while it should be checked? Is there something am I missing here ?

Note

The exception is not always reproducible since it only happens in production when there is a missing response from the Amazon service.

Update

I've verified down the stacktrace till reaching the AmazonHttpClient class whereas there is this code that is catching the `IOException' :

catch (IOException ioe) {
                if (log.isInfoEnabled()) {
                    log.info("Unable to execute HTTP request: " + ioe.getMessage(), ioe);
                }
                captureExceptionMetrics(ioe, awsRequestMetrics);
                awsRequestMetrics.addProperty(AWSRequestID, null);
                AmazonClientException ace = new AmazonClientException(
                        "Unable to execute HTTP request: " + ioe.getMessage(),
                        ioe);
                if (!shouldRetry(request.getOriginalRequest(),
                                p.apacheRequest,
                                ace,
                                p.requestCount,
                                config.getRetryPolicy())) {
                    throw lastReset(ace, request);
                }
                // Cache the retryable exception
                p.retriedException = ace;
            }

And the lastReset should be the responsible for the exception thrown, what I don't understand is how it is possible that the exception logged is org.apache.http.NoHttpResponseException ...

The line before the stacktrace is always :

2017-09-15 07:41:39 INFO  AmazonHttpClient:496 - Unable to execute HTTP request: sqs.eu-west-1.amazonaws.com failed to respond

Answer 1

My guess is that you're the victim of stack trace formatting.

I think you're right when you finger lastReset() as the culprit. This is where you see throws IOException disappear from the stack trace. This method is clearly throwing an AmazonClientException (a runtime exception), with the original NoHttpResponseException "inside" it.

You can simulate this with a snippet like so:

throw new AmazonClientException("Oh no!", new NoHttpResponseException("The AWS server doesn't like you"));

If I insert this line of code into an existing Java application (Spring Boot, in this case), this is what I see in my Eclipse console:

No sign of the AmazonClientException! Until, I scroll to the right:

Amazon made the decision to go with unchecked exceptions, and documented it here.

So they are indeed (I'm pretty sure) wrapping your IOException in a runtime exception, to "help" you by giving you "fine-grained control over the errors you handle", although this isn't always obvious.

All that said, I could be wrong. If I was right, I'd expect to see your custom SqsReceiverException at the top of the stack, since you do catch AmazonClientException.

It's hard to say for sure without the last few lines from standard out before the stack trace. If I'm off the mark, could you post them?

Update

The line you updated the question with (AmazonHttpClient:496) is the line printing the stack trace. When you pass a Throwable to log.info(), the stack trace will print. This trace is being logged before your exception is wrapped and re-thrown.

So this bit is seemingly being "swallowed":

com.amazonaws.AmazonClientException: Unable to execute HTTP request: sqs.us-east-1.amazonaws.com failed to respond
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:500)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
    at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2419)
    at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1130)
    at httptest.Main.main(Main.java:32)
Caused by:

and I can't speak to the missing SqsReceiverException. But I don't think the signature of DefaultRequestDirector.execute() is lying, and I don't think we're dealing with a compiler bug.

Perhaps you can add oe.printStackTrace() to your catch (AmazonClientException oe) block?

Finally, I'd suggest stepping through this with a debugger. To simulate your production issue, simply set a breakpoint at DefaultHttpResponseParser:140, and after this line executes, change i to -1. Then step back up the stack all the way back to your code.

I also set a breakpoint at AmazonHttpClient:971 so I could change retries and avoid going through the loop four times.