Problems using paho mqtt client with python 3.7

Question

I am running the following code to connect to a mqtt server.

import paho.mqtt.client as mqtt
import ssl
import uuid

client = mqtt.Client(str(uuid.uuid1()))
client.tls_set(
    "ca.crt",
    "client.crt",
    "client.key",
    cert_reqs=ssl.CERT_REQUIRED,
    tls_version=ssl.PROTOCOL_TLSv1
)
client.connect(
    "127.0.0.1",
    8883,
)
client.loop_forever()

This code works fine with python2.7 version. But when I run it with python3.7 version I am getting the below error.

Traceback (most recent call last):
  File "test.py", line 29, in <module>
    8883,
  File "virtualenvs/mqtt-xG2h6zri/lib/python3.7/site-packages/paho/mqtt/client.py", line 839, in connect
    return self.reconnect()
  File "mqtt-xG2h6zri/lib/python3.7/site-packages/paho/mqtt/client.py", line 994, in reconnect
    sock.do_handshake()
  File ".pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 1108, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '127.0.0.1'. (_ssl.c:1045)

Please help me figure out how to make this work in python 3.7.

Answer 1

Found the answer.

Actually, according to this link matching server IP address with CN field of certificate is deprecated for more than 15 years. But python versions lower than 3.7 still allow this even though it is deprecated. Therefore I had to create a certificate with the ip address of the server added in the SAN field.

Creating certificates with SAN fields is explained in this answer. But the solution in the answer uses domain names. If you are creating certificates with IP address use this command to create a certificate instead of the command in that answer.

openssl x509 -req -in server.csr \
        -extfile <(printf "subjectAltName=IP:127.0.0.1") \
        -CA ca.crt \
        -CAkey ca.key \
        -CAcreateserial -out server.crt \
        -days 365

After using these certificates the error is solved.

Answer 2

Had a similar problem (although using a RabbitsMQ client Pika rather than MQTT), but similar error being thrown in the SSL library upon do_handshake() method in python 3.8 when trying to connect to an external endpoint rather than localhost, and I could not generate a new ssl cert as it was provided to me.

The accepted answer helped me but I think it might help others, as downgrading to python3.6 gives a much more useful error message:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/miniconda/envs/venv3.6/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 359, in __init__
    self._impl = self._create_connection(parameters, _impl_class)
  File "/root/miniconda/envs/venv3.6/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 450, in _create_connection
    raise self._reap_last_connection_workflow_error(error)
  File "/root/miniconda/envs/venv3.6/lib/python3.6/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
    self._sock.do_handshake()
  File "/root/miniconda/envs/venv3.6/lib/python3.6/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/root/miniconda/envs/venv3.6/lib/python3.6/ssl.py", line 694, in do_handshake
    match_hostname(self.getpeercert(), self.server_hostname)
  File "/root/miniconda/envs/venv3.6/lib/python3.6/ssl.py", line 331, in match_hostname
    % (hostname, dnsnames[0]))
ssl.CertificateError: hostname '149.176.221.21' doesn't match 'redacted.internal.url'

where redacted.internal.url is what you have to set your hostname to for the connection (in ssl context object), as that is whatever is on your certificate.

This is regardless of what ip address/url you're actually connecting to, which i this case was 149.176.221.21