Python/sockets/ssl EOF occurred in violation of protocol

Question

I would like to authenticate the server at client's side in my echo client/server program. I'm using python 2.7.12 and the ssl module on

Distributor ID: Ubuntu
Description:    Ubuntu 14.04.5 LTS
Release:        14.04
Codename:       trusty

I've generated client's and server's certificates and keys using the openssl commands:

openssl req -new -x509 -days 365 -nodes -out client.pem -keyout client.key
openssl req -new -x509 -days 365 -nodes -out server.pem -keyout server.key

Versions of openssl library itself and openssl used by python are the same:

openssl version -a
OpenSSL 1.0.1f 6 Jan 2014
built on: Fri Sep 23 12:19:57 UTC 2016
platform: debian-amd64
options:  bn(64,64) rc4(16x,int) des(idx,cisc,16,int) blowfish(idx) 
compiler: cc -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
OPENSSLDIR: "/usr/lib/ssl"

python -c "import ssl; print ssl.OPENSSL_VERSION"
OpenSSL 1.0.1f 6 Jan 2014

However, the code below shows some errors, at server's side: EOF occurred in violation of protocol (_ssl.c:1645) (but the server still works), and at client's side:

Traceback (most recent call last):
  File "/http_ssl_client.py", line 36, in <module>
    if not cert or ('commonName', 'test') not in cert['subject'][4]: raise Exception("Invalid SSL cert for host %s. Check if this is a man-in-themiddle attack!" )
Exception: Invalid SSL cert for host %s. Check if this is a man-in-themiddle attack!
{'notBefore': u'Jun  3 11:54:21 2017 GMT', 'serialNumber': u'BBDCBEED69655B6E', 'notAfter': 'Jun  3 11:54:21 2018 GMT', 'version': 3L, 'subject': ((('countryName', u'pl'),), (('stateOrProvinceName', u'test'),), (('localityName', u'test'),), (('organizationName', u'test'),), (('organizationalUnitName', u'test'),), (('commonName', u'test'),), (('emailAddress', u'test'),)), 'issuer': ((('countryName', u'pl'),), (('stateOrProvinceName', u'test'),), (('localityName', u'test'),), (('organizationName', u'test'),), (('organizationalUnitName', u'test'),), (('commonName', u'test'),), (('emailAddress', u'test'),))}

Server's code:

#!/bin/usr/env python
import socket
import ssl

def main():
    HOST = '127.0.0.1'
    PORT = 1234

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind((HOST, PORT))
    sock.listen(5)

    while True:
        conn = None
        client_sock, addr = sock.accept()
        try:
            ssl_client = ssl.wrap_socket(client_sock, server_side=True, certfile="server.pem", keyfile="server.key", ssl_version=ssl.PROTOCOL_TLSv1_2)
            data =  ssl_client.read(1024)
            print data
            ssl_client.write(data)
        except ssl.SSLError as e:
            print(e)
        finally:
            if conn:
                conn.close()
if __name__ == '__main__':
    main()

Client:

#!/bin/usr/env python
import socket
import ssl

if __name__ == '__main__':

    HOST = '127.0.0.1'
    PORT = 1234

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((HOST, PORT))

    context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
    context.verify_mode = ssl.CERT_REQUIRED
    context.load_verify_locations('server.pem')

    if ssl.HAS_SNI:
        secure_sock = context.wrap_socket(sock, server_hostname=HOST)
    else:
        secure_sock = context.wrap_socket(sock)

    cert = secure_sock.getpeercert()
    print cert

    if not cert or ('commonName', 'test') not in cert['subject'][4]: raise Exception("Error" )

    secure_sock.write('hello')

    print secure_sock.read(1024)

    secure_sock.close()
    sock.close()

All files are in the same directory.

Answer 1

I'm going to walk through the issues I've spotted (although some are already spotted by someone else, and also fixed). Note that I've tried on Win 10 using Python 2.7.10, 2.7.13, 3.4.4, 3.5.3 and OpenSSL 1.0.2d, 1.0.2j (w/wo fips):

• server.py:

  • conn variable is initialized to None and that's it. The if clause under finally is useless. client_sock should probably be tested
  • Hint: the server socket (sock) could probably be wrapped (instead of client_sock); that way client_sock (returned by sock.accept()) would already be wrapped

• client.py:

  • The 1^st one is minor: the code in the snippet (raise Exception("Error")) is not the same as the one in the traceback (raise Exception("Invalid SSL cert for host %s. Check if this is a man-in-themiddle attack!"))

  • The way of searching for certificate attributes. As @TomaszPlaskota noticed, the commonName tuple index is wrong. Here's the pretty-printed certificate from my machine (the field values might/will differ):

    {
        'issuer': ((('countryName', 'AU'),),
                   (('stateOrProvinceName', 'Some-State'),),
                   (('localityName', 'CJ'),),
                   (('organizationName', 'Internet Widgits Pty Ltd'),),
                   (('organizationalUnitName', 'OU'),),
                   (('commonName', 'cfati-e5550-0'),),
                   (('emailAddress', '[email protected]'),)),
        'notAfter': 'Jun  5 17:21:03 2018 GMT',
        'notBefore': 'Jun  5 17:21:03 2017 GMT',
        'serialNumber': 'C4A03B2BE4F959A9',
        'subject': ((('countryName', 'AU'),),
                    (('stateOrProvinceName', 'Some-State'),),
                    (('localityName', 'CJ'),),
                    (('organizationName', 'Internet Widgits Pty Ltd'),),
                    (('organizationalUnitName', 'OU'),),
                    (('commonName', 'cfati-e5550-0'),),
                    (('emailAddress', '[email protected]'),)),
        'version': 3
    }
    

    There might be cases (at least theoretically) when the certificate would be incomplete. A more robust form of the check would be:

    def check_certificate(cert, field=("commonName", "test")):
        if not cert:
            return False
        for pairs in cert.get("subject", ()):
            if field in pairs:
                return True
        return False
    
    # ....
    
    if not check_certificate(cert):
        raise Exception("Error")
    

  • The way of handling errors. If the server certificate is not OK, you simply raise an error which breaks the communication and exits the client program abnormally. That fact, on the server side triggers ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:590) (from your other question: [SO]: Mutual ssl authentication in simple ECHO client/server [Python / sockets / ssl modules], ssl.SSLEOFError: EOF occurred in violation of protocol), when the server tries to write back to the client (ssl_client.write(data)).
    Although in some cases at program termination the OS cleans up the resources, it's recommended to always do the cleanup from code. So, instead of just raising an exception, do something like (as @JamesKPolk suggested):

    plain_sock = secure_sock.unwrap()
    plain_sock.shutdown(socket.SHUT_RDWR)
    plain_sock.close()
    # Any other cleanup action here
    

• The biggest problem is that you are using 2 self-signed certificates (that have nothing to each other). A certificate being self-signed means:

  • It's its own signer (or parent). That's why the Issuer and Subject fields are the same (or to be more rigorous: its Issuer Key Identifier and Subject Key Identifier extensions are the same)
  • It's a CA (Certificate Authority) certificate (if looking in its Basic Constraints, you'll notice Subject Type is CA). This is like a consequence of the former bullet

Few words about certificates: they are organized in trees: meaning that there will be a root node also known as Root CA certificate. That node might have several child nodes (a child node means that it's signed by its parent). Those nodes can also be CA or end user (EU) certs. Each CA cert might also have children, and there is our tree. Its leafs are EU certificates. The path between the Root CA and a leaf certificate (composed by the Root CA, Intermediate CAs and EU) is called a certificate chain.
A self-signed cert can be represented as a tree consisting of a single node. Note that being a CA cert (as in our case) it can also be used to sign other certs (it can have children).

Certificate validation - when a certificate is verified to make sure that it is "who" it claims it is. That is done by checking it against its parent CA (and recursively all the CAs in the chain are checked til the Root CA is reached). If everything is OK, then the certificate is valid. This is of course a very simplistic version, the subject is quite complex, but there's lots of info that can be found on the Internet. Needless to say that the entity that performs the validation needs to have access to the CAs in the chain. You can take the Web browser as an example, its certificate "vault" contains:

• Optionally: certificates (EU) that were supplied to you in order to be able to connect to some sites (e.g. Personal Certificates)
• Certificates (CA) required to validate other certificates presented by web servers (Trusted Root/Intermediate Certification Authorities)

[IBM]: An overview of the SSL or TLS handshake (although there are many other places) briefly describes how SSL connection works.
Typically a secure system, will emit a (unique) certificate (EU) for each of its clients; that certificate will be bond to the client machine (*IP8 address or FQDN); here CRL and OCSP worth being mentioned.

Back to the question: because we have a particular situation (2 certs where each is CA, but they are also used as EUs), it may not be so obvious, but I'll do my best to explain. Considering that certificate validation will happen at both communication ends (two-way), both certificates must be loaded in both applications. For example, in server app:

• server certificate must be loaded in order to be presented to any client that connects, in order for the client to verify it
• client certificate must be loaded (as a CA) in order to verify the certificate that any client presents (which will be the same certificate). Take look at [Python 2.Docs]: ssl.wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, ssl_version={see docs}, ca_certs=None, do_handshake_on_connect=True, suppress_ragged_eofs=True, ciphers=None)(ca_certs argument)

Obviously, for client, it will be the other way around.