Axis exception: o.a.axis2.transport.http.HTTPSender.sendViaPost(196) - Unable to sendViaPost - Connection refused

Question

I have a server that is executing a Webservice call to an external server. This call has to occur over SSL and using a proxy:

My truststore is well configured:

trustStore is: /opt/configuration/keystore/truststore.jks
trustStore type is : jks
trustStore provider is : 
init truststore
adding as trusted cert:

Also my handshake is done correctly:

https-jsse-nio-8443-exec-6, READ: TLSv1.2 Handshake, length = 40
Padded plaintext after DECRYPTION:  len = 16
0000: 14 00 00 0C 78 49 E5 E1   29 04 A5 1D FC 4F E6 E2  ....xI..)....O..
*** Finished
verify_data:  { 120, 73, 229, 225, 41, 4, 165, 29, 252, 79, 230, 226 }
***
[read] MD5 and SHA1 hashes:  len = 16
0000: 14 00 00 0C 78 49 E5 E1   29 04 A5 1D FC 4F E6 E2  ....xI..)....O..
https-jsse-nio-8443-exec-6, WRITE: TLSv1.2 Change Cipher Spec, length = 1
[Raw write]: length = 6
0000: 14 03 03 00 01 01                                  ......
*** Finished

I have a ProxySelector that selects a proxy to use:

@Override
public List<Proxy> select(URI uri) {

    if (LOGGER.isDebugEnabled()) {
        LOGGER.debug("select for URL : " + uri);
    }

    if (uri == null) {
        throw new IllegalArgumentException("URI cannot be null.");
    }

    // Get protocol and management configuration for HTTP or HTTPS.
    String protocol = uri.getScheme();
    if (StringUtils.equalsIgnoreCase("http", protocol)
            || StringUtils.equalsIgnoreCase("https", protocol)
            || StringUtils.equalsIgnoreCase("socket", protocol)) {
        LOGGER.debug("Retrieving proxy list...");
        List<Proxy> proxyList = new ArrayList<>();
        for (InnerProxy p : proxies.values()) {

            proxyList.add(p.toProxy());
        }
        LOGGER.debug(proxyList.size() + " configured proxies");
        // Return configured Proxy
        return proxyList;
    }

    /*
     * For others protocols (could be SOCKS or FTP etc.) return the default
     * selector.
     */
    if (defaultSelector != null) {
        return defaultSelector.select(uri);
    } else {
        List<Proxy> proxyList = new ArrayList<>();
        proxyList.add(Proxy.NO_PROXY);
        return proxyList;
    }
}

But when calling a method on a webmethod generated by JAX-WS RI:

/**
 * 
 * @param base64ObjectToValidate
 * @param xmlMetadata
 * @param xmlReferencedStandard
 * @return
 *     returns java.lang.String
 * @throws SOAPException_Exception
 */
@WebMethod
@WebResult(name = "validationResult", targetNamespace = "")
@RequestWrapper(localName = "validateObject", targetNamespace = "http://ws.validator.sch.gazelle.ihe.net/", className = "net.ihe.gazelle.schematron.ValidateObject")
@ResponseWrapper(localName = "validateObjectResponse", targetNamespace = "http://ws.validator.sch.gazelle.ihe.net/", className = "net.ihe.gazelle.schematron.ValidateObjectResponse")
public String validateObject(
    @WebParam(name = "base64ObjectToValidate", targetNamespace = "")
    String base64ObjectToValidate,
    @WebParam(name = "xmlReferencedStandard", targetNamespace = "")
    String xmlReferencedStandard,
    @WebParam(name = "xmlMetadata", targetNamespace = "")
    String xmlMetadata)
    throws SOAPException_Exception
;

I get 8 query select requests (where I would only expect 1):

15:55:30.228+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : https://gazelle.ehdsi.ihe-europe.net/SchematronValidator-ejb/GazelleObjectValidatorService/GazelleObjectValidator?wsdl
15:55:30.495+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : https://gazelle.ehdsi.ihe-europe.net/SchematronValidator-ejb/GazelleObjectValidatorService/GazelleObjectValidator?wsdl
15:55:30.637+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : https://gazelle.ehdsi.ihe-europe.net/SchematronValidator-ejb/GazelleObjectValidatorService/GazelleObjectValidator?wsdl
15:55:30.667+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : https://gazelle.ehdsi.ihe-europe.net/SchematronValidator-ejb/GazelleObjectValidatorService/GazelleObjectValidator?wsdl
15:55:31.130+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : socket://gazelle.ehdsi.ihe-europe.net:443
15:55:31.134+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : socket://gazelle.ehdsi.ihe-europe.net:443
15:55:31.137+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : socket://gazelle.ehdsi.ihe-europe.net:443
15:55:31.140+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : socket://gazelle.ehdsi.ihe-europe.net:443   

The Custom Proxy selector is called 4 times for the real URL and 4 times for the socket.

And in the logs I got following exception caused by a HTTP request initiated by Axis (I'm using Axis version 1.6.2):

15:55:31.137+02:00 [https-jsse-nio-8443-exec-6] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : socket://gazelle.ehdsi.ihe-europe.net:443
15:55:31.139+02:00 [https-jsse-nio-8443-exec-6] DEBUG o.a.c.httpclient.HttpMethodDirector.executeWithRetry(404) - Closing the connection.
15:55:31.139+02:00 [https-jsse-nio-8443-exec-6] INFO  o.a.c.httpclient.HttpMethodDirector.executeWithRetry(439) - I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
15:55:31.139+02:00 [https-jsse-nio-8443-exec-6] DEBUG o.a.c.httpclient.HttpMethodDirector.executeWithRetry(443) - Connection refused (Connection refused)
    java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:140)
        at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:130)
        at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
        at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
        at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
        at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
        at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:621)
        at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:193)
        at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
        at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:404)
        at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:231)
        at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:443)
        at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:406)
        at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
        at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
        at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.execute(AxisInvocationController.java:578)
        at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.doInvoke(AxisInvocationController.java:127)
        at org.apache.axis2.jaxws.core.controller.impl.InvocationControllerImpl.invoke(InvocationControllerImpl.java:93)
        at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invokeSEIMethod(JAXWSProxyHandler.java:373)
        at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invoke(JAXWSProxyHandler.java:171)
        at com.sun.proxy.$Proxy137.validateObject(Unknown Source)

I have no idea why Axis executes these calls and why I have a java.net.ConnectException: Connection refused (Connection refused) exception...

Looks like he doesn't use the configured proxy...

Hope anyone could help me out...

I found following lines in my logging:

16:35:35.676+02:00 [https-jsse-nio-8443-exec-5] DEBUG o.a.c.httpclient.HttpConnection.open(692) - Open connection to gazelle.ehdsi.ihe-europe.net:443
16:35:35.677+02:00 [https-jsse-nio-8443-exec-5] DEBUG e.epsos.util.net.CustomProxySelector.select(43) - select for URL : socket://gazelle.ehdsi.ihe-europe.net:443
16:35:35.677+02:00 [https-jsse-nio-8443-exec-5] DEBUG e.epsos.util.net.CustomProxySelector.select(52) - protocol is : socket
16:35:35.677+02:00 [https-jsse-nio-8443-exec-5] DEBUG e.epsos.util.net.CustomProxySelector.select(56) - Retrieving proxy list...
16:35:35.677+02:00 [https-jsse-nio-8443-exec-5] DEBUG e.epsos.util.net.CustomProxySelector.select(62) - 1 configured proxies
16:35:35.680+02:00 [https-jsse-nio-8443-exec-5] DEBUG o.a.c.httpclient.HttpMethodDirector.executeWithRetry(404) - Closing the connection.
16:35:35.680+02:00 [https-jsse-nio-8443-exec-5] DEBUG o.a.c.httpclient.HttpMethodDirector.executeWithRetry(434) - Method retry handler returned false. Automatic recovery will not be attempted
16:35:35.680+02:00 [https-jsse-nio-8443-exec-5] DEBUG o.a.c.httpclient.HttpConnection.releaseConnection(1178) - Releasing connection back to connection manager.
16:35:35.680+02:00 [https-jsse-nio-8443-exec-5] DEBUG o.a.c.h.MultiThreadedHttpConnectionManager.freeConnection(979) - Freeing connection, hostConfig=HostConfiguration[host=https://gazelle.ehdsi.ihe-europe.net]
16:35:35.680+02:00 [https-jsse-nio-8443-exec-5] DEBUG o.a.c.h.util.IdleConnectionHandler.add(76) - Adding connection at: 1503930935680
16:35:35.681+02:00 [https-jsse-nio-8443-exec-5] DEBUG o.a.c.h.MultiThreadedHttpConnectionManager.notifyWaitingThread(961) - Notifying no-one, there are no waiting threads
16:35:35.681+02:00 [https-jsse-nio-8443-exec-5] INFO  o.a.axis2.transport.http.HTTPSender.sendViaPost(196) - Unable to sendViaPost to url[https://gazelle.ehdsi.ihe-europe.net/SchematronValidator-ejb/GazelleObjectValidatorService/GazelleObjectValidator]
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Answer 1

I think the reason why it doesn't work is explained in this bugreport. It's 11 years old and still marked as unresolved. The comment clearly says that there are no plans to implement that but there might be a workaround:

However, Axis1.x and 2.x can both switch to use the jakarta-commons httpclient library. This is what you should be using if you can, because it understads HTTP properly. If you could get proxy support in there, either axis build could pick it up.

As far as I remember, the commons HTTP client supports ProxySelector.