Using Pentaho Kettle, how do I automatically retry rest requests which fail due to connection hiccups?

Question

How can we make Pentaho retry rest requests on connection errors?

We have a Pentaho BI system which, among numerous data sources, is querying a particular REST api for over 20k query variations each run.

Predicatably, on most runs a few of these requests will fail due to connection hiccups. These usually manifest themselves in the Pentaho log as org.pentaho.di.core.exception.KettleException due to javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake, ultimately caused by java.io.EOFException: SSL peer shut down incorrectly.

After scouring the internet and Pentaho forums, we cannot find any instructions on adding a simple retry approach in cases such as this. Hair is being torn out.

Help me StackOverflow, you're my only hope!

Update 1: Stacktrace:

ERROR 29-11 11:02:17,659 - B - org.pentaho.di.core.exception.KettleException:
Can not result from [https://<DOMAIN>/<PATH>?<PARAMS>]
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake    
    at org.pentaho.di.trans.steps.rest.Rest.callRest(Rest.java:190)
    at org.pentaho.di.trans.steps.rest.Rest.processRow(Rest.java:385)
    at org.pentaho.di.trans.step.RunThread.run(RunThread.java:40)
    at java.lang.Thread.run(Thread.java:662)
Caused by: com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:213)
    at com.sun.jersey.client.apache.ApacheHttpClientHandler.handle(ApacheHttpClientHandler.java:175)
    at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:81)
    at com.sun.jersey.api.client.Client.handle(Client.java:648)
    at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
    at com.sun.jersey.api.client.WebResource.get(WebResource.java:191)
    at org.pentaho.di.trans.steps.rest.Rest.callRest(Rest.java:141)
    ... 3 more
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:632)
    at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
    at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(HttpConnection.java:827)
    at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.flushRequestOutputStream(MultiThreadedHttpConnectionManager.java:1525)
    at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:1975)
    at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:993)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
    at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:210)
    ... 9 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
    ... 22 more

Answer 1

or use step error handling on the rest client step, and direct the error fields to another rest client step.

obviously that only lets you retry as many times as you have steps, but there's no point retrying more than once anyway.