SSL issues on Python

Question

I have a code in Python that accepts (E)SMTP requests via aiosmtp but since I pushed this code on Debian 10, I'm having a few errors that wasn't present before (and my code didn't changed):

[SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)

SSL handshake failed protocol: transport: <_SelectorSocketTransport fd=11 read=polling write=>

SSLError: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)
  File "asyncio/sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()

and:

[SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)

SSL error in data received protocol: transport: <_SelectorSocketTransport fd=15 read=polling write=>

SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)
  File "asyncio/sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "asyncio/sslproto.py", line 207, in feed_ssldata
    self._sslobj.unwrap()
  File "ssl.py", line 767, in unwrap
    return self._sslobj.shutdown()

I think these two issues are related.

Unfortunately the two stacktrace don't show anything related to my code, which makes it harder for me to better see where this is happening, and the exception isn't related to another exception (Python3).

---

Here's the versions of my package:

uname -a : Linux my-server 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5+deb10u2 (2019-08-08) x86_64 GNU/Linux

python --version: Python 3.7.3

pip freeze

aiomysql==0.0.20
aiosmtpd==1.2
asn1crypto==0.24.0
atpublic==1.0
authres==1.2.0
beanstalkc3==0.4.0
blinker==1.4
certifi==2018.8.24
cffi==1.12.3
chardet==3.0.4
Click==7.0
cloud-init==18.3
configobj==5.0.6
cryptography==2.6.1
distro-info==0.21
dkimpy==0.9.4
dnspython==1.16.0
fail2ban==0.10.2
Flask==1.1.1
idna==2.6
itsdangerous==1.1.0
Jinja2==2.10.1
jsonpatch==1.21
jsonpointer==1.10
jsonschema==2.6.0
MarkupSafe==1.1.0
mysqlclient==1.4.4
oauthlib==2.1.0
psutil==5.6.3
py3dns==3.2.1
pycparser==2.19
PyGObject==3.30.4
pyinotify==0.9.6
PyJWT==1.7.0
PyMySQL==0.9.2
PyNaCl==1.3.0
pyspf==2.0.13
pysrs==1.0.3
python-apt==1.8.4
python-dotenv==0.10.3
PyYAML==3.13
requests==2.21.0
sentry-sdk==0.12.3
six==1.12.0
systemd-python==234
unattended-upgrades==0.1
urllib3==1.24.1
uWSGI==2.0.18
Werkzeug==0.16.0

---

I believe that if something was wrong with my code, I would have had this error on Debian 9 and earlier which I never had.

I searched on SO and Google about this error but didn't find anything. I suspect some issue on a specific version of a specific project (aiosmtpd, async or python) but don't have any clue.

I'm hoping you'll be able to help me :)

---

Update:

I've added tracking of ciphers in the communication. The shared ciphers are:

[[TLS_AES_256_GCM_SHA384, TLSv1.3, 256], [TLS_CHACHA20_POLY1305_SHA256, TLSv1.3, 256], [TLS_AES_128_GCM_SHA256, TLSv1.3, 128], [ECDHE-ECDSA-AES256-GCM-SHA384, TLSv1.2, 256], [ECDHE-RSA-AES256-GCM-SHA384, TLSv1.2, 256], [DHE-RSA-AES256-GCM-SHA384, TLSv1.2, 256], [ECDHE-ECDSA-CHACHA20-POLY1305, TLSv1.2, 256], [ECDHE-RSA-CHACHA20-POLY1305, TLSv1.2, 256], [DHE-RSA-CHACHA20-POLY1305, TLSv1.2, 256], [ECDHE-ECDSA-AES128-GCM-SHA256, TLSv1.2, 128], [ECDHE-RSA-AES128-GCM-SHA256, TLSv1.2, 128]]

And the Cipher for the socket is: [ECDHE-RSA-AES256-GCM-SHA384, TLSv1.2, 256] which is in the shared ciphers.

---

Update 2

I can reproduce the error, but only under specific conditions.

On the new server, here's the code I run:

import asyncio, logging, sys, signal, ssl
from aiosmtpd.controller import Controller
from aiosmtpd.handlers import Debugging
from aiosmtpd.smtp import SMTP

class ControllerTls(Controller):
    def factory(self):
        context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
        context.load_cert_chain('./certs/certificate.pem', './certs/id_rsa')
        context.load_dh_params('./certs/dhparams.pem')
        return SMTP(
            self.handler,
            tls_context=context
        )


# Temporary outputing errors from mail.log
streamHandler = logging.StreamHandler(sys.stdout)
streamHandler.setFormatter(logging.Formatter('[%(asctime)-15s] (%(levelname)s) - %(message)s'))
streamHandler.setLevel(logging.INFO)

maillog = logging.getLogger('mail.log')
maillog.setLevel(logging.INFO)
maillog.addHandler(streamHandler)

controller = ControllerTls(Debugging(), hostname='0.0.0.0', port=2125)
controller.start()
print('Controller started!')
sig = signal.sigwait([signal.SIGINT, signal.SIGQUIT])
controller.stop()

It's a basic script that helps me reproduce the issue.

On the Old server, I run this code:

import smtplib, ssl, sys

port = 25
if len(sys.argv) == 3:
    port = sys.argv[2]

def com(client, command, *args, **kwargs):
    result = getattr(client, command)(*args, **kwargs)
    if result[0] > 500:
        print('[FATAL] - An error occured!')
        print(result)
        client.quit()
        exit()

context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain('/var/www/towboat/certs/certificate.pem', '/var/www/towboat/certs/id_rsa')
context.set_ciphers('ECDHE-ECDSA-AES256-GCM-SHA384')
client = smtplib.SMTP(sys.argv[1], port=port)
com(client, 'ehlo')
com(client, 'starttls', context=context)
com(client, 'ehlo')
com(client, 'mail', '[email protected]')
com(client, 'rcpt', '[email protected]')
com(client, 'quit')

print('All good !')

Which I call with :

sendmail.py {ip.of.new.server} 2125

On the old server (the one running the script), I get this error:

Controller started!
[2019-10-08 15:57:11,878] (INFO) - Peer: ('ip.of.old.server', 45492)
[2019-10-08 15:57:11,878] (INFO) - ('ip.of.old.server', 45492) handling connection
[2019-10-08 15:57:11,880] (INFO) - ('ip.of.old.server', 45492) Data: b'ehlo {name old server}'
[2019-10-08 15:57:11,883] (INFO) - ('ip.of.old.server', 45492) Data: b'STARTTLS'
[2019-10-08 15:57:11,883] (INFO) - ('ip.of.old.server', 45492) STARTTLS
SSL handshake failed
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f04d33d7d30>
transport: <_SelectorSocketTransport fd=7 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f04d33d7d30>
transport: <_SelectorSocketTransport closing fd=7 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:1056)
[2019-10-08 15:58:33,909] (INFO) - Connection lost during _handle_client()

What's super odd, is that if I copy the sendmail script on my local machine, and run it pointing to the new server, I don't have the error anymore!

(So the issue must be related to the old server? But why the new server shows the exception?!)

If I switch the scripts (testing sending an email from the new server to the old), it works...

Answer 1

I think that is the cause:

v1.1.1d on the new server, 1.1.0d on the old one

The 1.1.1 line introduces the TLSv3 and many other quite important changes - see the change log.

As I have seen you opening a ticket at aiosmtpd github you have guessed correctly that the reason why you are getting the error is the aiosmtpd. The reason being there is it supports You need at least Python 3.5, which has no support for openssl 1.1.1. Only python 3.7 (it has not been fully backported even to python 3.6) currently supports openssl 1.1.1.

Since the latest version of aiosmtpd is 1.2 (2018-09-01) it is save to assume (did not see any PR(s) for that) that they have not yet implemented the new openssl 1.1.1 [11 Sep 2018] which is introduces major changes.

Your only option, beside providing a PR for aiosmtpd, is to downgrade your openssl to the latest of the 1.1.0 line which is currently 1.1.0i.

Answer 2

I see at first that cipher ECDHE-ECDSA-AES256-GCM-SHA384 cannot work because both sides using RSA certificates (I'm wondering whether client really uses his certificate for authentication or whether it's only wrongly configured in server mode).
If You run the same client script on a python working with OpenSSL 1.1.1 the servers will agree to TLSv1.3, where the cipher suites cannot be disabled anymore so are still allowed even with the set_ciphers.
I assume if You choose proper ECDHE-RSA-AES256-GCM-SHA384 or change nothing at all in that regard on Your "old" (Openssl 1.1.0) Debian 9 it will connect without issues with TLSv1.2 to the new server.

That said, the issue with the old script to determine cipher-suites on new server is also related to the fact, that the TLSv1.3 cipher suites cannot be disabled and the script expects it can disable any cipher suite for the test (that is the way it works).

There are some cipher suites that are now completely out of OpenSSLv1.1.1, but there are others that are just disabled by default (current Python only allows HIGH ciphers and no MD5/RC4 by default - and no ciphers without athentication - and of course no SSLv3 and older).
Since python 3.6 it's very simple to get the list of offered ciphers (so the broken script is not needed anymore):

root@somehost:~# python3
Python 3.7.3 (default, Oct  7 2019, 12:56:13)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
>>> for cipher in ctx.get_ciphers(): print(cipher['name']+' '+cipher['protocol']) if cipher['auth'] == 'auth-rsa' else None
...
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2
DHE-RSA-AES256-GCM-SHA384 TLSv1.2
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2
DHE-RSA-CHACHA20-POLY1305 TLSv1.2
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
DHE-RSA-AES128-GCM-SHA256 TLSv1.2
ECDHE-RSA-AES256-SHA384 TLSv1.2
DHE-RSA-AES256-SHA256 TLSv1.2
ECDHE-RSA-AES128-SHA256 TLSv1.2
DHE-RSA-AES128-SHA256 TLSv1.2
ECDHE-RSA-AES256-SHA TLSv1.0
DHE-RSA-AES256-SHA SSLv3
ECDHE-RSA-AES128-SHA TLSv1.0
DHE-RSA-AES128-SHA SSLv3
AES256-GCM-SHA384 TLSv1.2
AES128-GCM-SHA256 TLSv1.2
AES256-SHA256 TLSv1.2
AES128-SHA256 TLSv1.2
AES256-SHA SSLv3
AES128-SHA SSLv3

So fo a TLSv1.2 connection the minimum requirement is AES128-SHA256 (without [EC]DHE KEX) or DHE-RSA-AES128-SHA256/ECDHE-RSA-AES128-SHA256 / minimum OpenSSL Version 1.0.1 Released 14-March-2012. So not OpenSSL version supporting TLSv1.2 should be unable to connect.
In production, thought, there may be some OpenSSLv0.98 clients (at least I know some I have still to maintain - thought I have all Tools I need for production build against a newer OpenSSL). These can only talk TLSv1.0 if not the "forbidden" SSLv3. They could at least use the suites shown as SSLv3.

So there shouldn't be many clients that cannot use one of the default proposals, but still there could be some that are eigther configured for specific cipher suites that are not preferrable from todays point of view, or even older systems, less powerfull hardware, different SSL library vendor, ... So You need to find an example that is really not working to see which additional cyphersuite has to be allowed - or maybe different authentication mechanism.

For the other point in this discussion like downgrading OpenSSL, that is definitly not the way to go. There cannot be many clients that cannot connect with default setting and even less that cannot connect at all with all available settings OpenSSLv1.1.1 still offers. And if there are they need an update, definitly..

If You really wanted/needed You can compile an older OpenSSL to go in a distinct location and a Python against that older OpenSSL version. Maybe start a second server on a second port just for that clients. Or You can run a container instead for that, but You cannot downgrade the system-OpenSSL.

The 2nd error, KRB5_S_INIT, is indeed a bug. It looks like a bug in python-core async module which was introduced with Python 7.3. But this error happens only if the connection is anyway unusable (so after the connection was aandoned because of the no common cipher suites case).