Protocol error between Android and Python using SSL

Question

This question has been asked, in various flavors, several times. Unfortunately, none of the answers have yielded a solution for me.

I am attempting to connect to a python web server (code to follow) using the https protocol, with client side authentication included. When I connect via a python client that I wrote for testing, I have no problems. Fast forward, I am now trying to connect from an Android device (code to follow again) and am getting a

javax.net.ssl.SSLProtocolException

I have a self signed CA, which has issued two certificates. One for the client and one for the server.

I removed the passphrase from the server private key using:

openssl rsa -in serverKey.pem -out serverKey.pem

and issued a request using openssl from the linux command line.

For the client I issued a request, created the certificate and then used keytool with the BouncyCastle provider to import the CA into a trust store and the client certificate into a key store (I realize they are the same format, it just helps me to keep them separated if I refer to them by different names).

Relevant server code:

class ReuseHTTPServer(BaseHTTPServer.HTTPServer):
    def __init__(self, address, handler):
        BaseHTTPServer.HTTPServer.__init__(self, address, handler)

        self.address = address

        ssl_socket = ssl.wrap_socket(socket.socket(self.address_family,
                                                   self.socket_type),
                                     keyfile = KEY_PATH,
                                     certfile = CERTIFICATE_PATH,
                                     server_side = True,
                                     cert_reqs = ssl.CERT_REQUIRED,
                                     ssl_version = ssl.PROTOCOL_SSLv23,
                                     ca_certs = CA_PATH)
        s = self.socket.getsockname()

        print "serving:", s[0], "on port:", s[1]

        self.socket = ssl_socket

        self.server_bind()
        self.server_activate()

    def server_bind(self):
        BaseHTTPServer.HTTPServer.server_bind(self)

Android client code:

//add bouncy castle to the list of security providers
Security.insertProviderAt(new BouncyCastleProvider(), 1);

//load the trusted CA
KeyStore trusted = KeyStore.getInstance("BKS");
InputStream in = getResources().openRawResource(R.raw.mobilecastore);
trusted.load(in, "password".toCharArray());
in.close();
TrustManagerFactory trust_factory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trust_factory.init(trusted);

//load the client keystore
KeyStore client = KeyStore.getInstance(KeyStore.getDefaultType());
InputStream client_in = getResources().openRawResource(R.raw.client);
client.load(client_in, "password2".toCharArray());
client_in.close();
KeyManagerFactory key_factory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());

key_factory.init(client, "password2".toCharArray());

SSLContext ssl_context = SSLContext.getInstance("SSL"); 
ssl_context.init(key_factory.getKeyManagers(), trust_factory.getTrustManagers(), null);

URL url = new URL("https", IP, 60000, "/cgi-bin/www_sel_jf");
connection = (HttpsURLConnection) url.openConnection();
connection.setSSLSocketFactory(ssl_context.getSocketFactory());
connection.setDoInput(true);
connection.setRequestMethod("GET");
connection.connect();

The error:

12-31 07:23:37.917: W/System.err(3666): javax.net.ssl.SSLHandshakeException: javax.net.ssl.SSLProtocolException: SSL handshake terminated: ssl=0x16bab18: Failure in SSL library, usually a protocol error
12-31 07:23:37.917: W/System.err(3666): error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (external/openssl/ssl/s3_pkt.c:1234 0x16bf980:0x00000003)
12-31 07:23:37.917: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:460)
12-31 07:23:37.917: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:257)
12-31 07:23:37.917: W/System.err(3666):     at libcore.net.http.HttpConnection.setupSecureSocket(HttpConnection.java:210)
12-31 07:23:37.917: W/System.err(3666):     at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.makeSslConnection(HttpsURLConnectionImpl.java:477)
12-31 07:23:37.917: W/System.err(3666):     at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:441)
12-31 07:23:37.917: W/System.err(3666):     at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:282)
12-31 07:23:37.917: W/System.err(3666):     at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:232)
12-31 07:23:37.917: W/System.err(3666):     at libcore.net.http.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:80)
12-31 07:23:37.917: W/System.err(3666):     at libcore.net.http.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:164)
12-31 07:23:37.917: W/System.err(3666):     at com.dsndata.sds2mobile.status.activities.ServerJobs$RetrieveJobNames.doInBackground(ServerJobs.java:143)
12-31 07:23:37.917: W/System.err(3666):     at com.dsndata.sds2mobile.status.activities.ServerJobs$RetrieveJobNames.doInBackground(ServerJobs.java:1)
12-31 07:23:37.917: W/System.err(3666):     at android.os.AsyncTask$2.call(AsyncTask.java:264)
12-31 07:23:37.925: W/System.err(3666):     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
12-31 07:23:37.925: W/System.err(3666):     at java.util.concurrent.FutureTask.run(FutureTask.java:137)
12-31 07:23:37.925: W/System.err(3666):     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
12-31 07:23:37.925: W/System.err(3666):     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
12-31 07:23:37.925: W/System.err(3666):     at java.lang.Thread.run(Thread.java:856)
12-31 07:23:37.925: W/System.err(3666): Caused by: javax.net.ssl.SSLProtocolException: SSL handshake terminated: ssl=0x16bab18: Failure in SSL library, usually a protocol error
12-31 07:23:37.925: W/System.err(3666): error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (external/openssl/ssl/s3_pkt.c:1234 0x16bf980:0x00000003)
12-31 07:23:37.925: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_do_handshake(Native Method)
12-31 07:23:37.925: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:410)
12-31 07:23:37.925: W/System.err(3666):     ... 16 more
12-31 07:23:37.925: W/System.err(3666): javax.net.ssl.SSLHandshakeException: javax.net.ssl.SSLProtocolException: SSL handshake terminated: ssl=0x16bab18: Failure in SSL library, usually a protocol error
12-31 07:23:37.925: W/System.err(3666): error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (external/openssl/ssl/s3_pkt.c:1234 0x16bf980:0x00000003)
12-31 07:23:37.925: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:460)
12-31 07:23:37.925: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:257)
12-31 07:23:37.925: W/System.err(3666):     at libcore.net.http.HttpConnection.setupSecureSocket(HttpConnection.java:210)
12-31 07:23:37.925: W/System.err(3666):     at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.makeSslConnection(HttpsURLConnectionImpl.java:477)
12-31 07:23:37.925: W/System.err(3666):     at libcore.net.http.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:441)
12-31 07:23:37.925: W/System.err(3666):     at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:282)
12-31 07:23:37.925: W/System.err(3666):     at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:232)
12-31 07:23:37.925: W/System.err(3666):     at libcore.net.http.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:80)
12-31 07:23:37.925: W/System.err(3666):     at libcore.net.http.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:164)
12-31 07:23:37.925: W/System.err(3666):     at com.dsndata.sds2mobile.status.activities.ServerJobs$RetrieveJobNames.doInBackground(ServerJobs.java:143)
12-31 07:23:37.932: W/System.err(3666):     at com.dsndata.sds2mobile.status.activities.ServerJobs$RetrieveJobNames.doInBackground(ServerJobs.java:1)
12-31 07:23:37.932: W/System.err(3666):     at android.os.AsyncTask$2.call(AsyncTask.java:264)
12-31 07:23:37.932: W/System.err(3666):     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
12-31 07:23:37.932: W/System.err(3666):     at java.util.concurrent.FutureTask.run(FutureTask.java:137)
12-31 07:23:37.932: W/System.err(3666):     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
12-31 07:23:37.932: W/System.err(3666):     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
12-31 07:23:37.932: W/System.err(3666):     at java.lang.Thread.run(Thread.java:856)
12-31 07:23:37.932: W/System.err(3666): Caused by: javax.net.ssl.SSLProtocolException: SSL handshake terminated: ssl=0x16bab18: Failure in SSL library, usually a protocol error
12-31 07:23:37.932: W/System.err(3666): error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (external/openssl/ssl/s3_pkt.c:1234 0x16bf980:0x00000003)
12-31 07:23:37.932: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_do_handshake(Native Method)
12-31 07:23:37.932: W/System.err(3666):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:410)
12-31 07:23:37.932: W/System.err(3666):     ... 16 more

It should be noted that I am using the javax.net.ssl library and not the apache one.

EDIT: The python server is hosted on a Windows machine. Is there any extra configuration that needs to be taken into account for ssl on a Windows port?

EDIT: Yes, there is something that needs to be done to allow SSL on a windows port. Switched to port 443 (designated for SSL traffic) and am making (slow) progress.

EDIT: I am now able to track the request using Wireshark (I am learning a lot for this problem!) and Wireshark is telling me that there is a 405 error. Which in SSL means that the certificate format is not recognized. The python server is using PEM certificates (the only format allowed according to python docs) and the certificates imported into the keystore on the Android device are DER (to my knowledge the only format accepted by BKS).

Any help would be greatly appreciated.

Answer 1

This can be a SSL/TLS version or algorithms mismatch error. Your server uses SSL v.2-3, while this is quite old and TLS 1.0-1.2 should be used. The best way to debug this is to run Wireshark, and see which client and server's SSL/TLS handshake packets are sent and when connection is dropped.

Answer 2

After learning how to read the packets, it turns out that the issue was with my certificates. The common name was not matched to the URL.