How to configure Python to ignore the hostname verification?

Question

We are relatively new to in Python therefore may be the question is too simple. We are using Python version 2.7.15.

We are trying to use Python over TLS without success.

This is our code:

import ssl,socket
import urllib2

context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = False
context.load_verify_locations("/py-test/python/bin/certificate.pem")
url = "https://10.0.0.12"
request = urllib2.Request(url)
websocket = urllib2.urlopen(request,None,None,None,None,None,context)
pages=websocket.readlines()
print pages

As you see, we have configured context.check_hostname = False

Unfortunately, it fails with the following exception

Traceback (most recent call last):
 File "./test.py", line 11, in <module>
   websocket = urllib2.urlopen(request,None,None,None,None,None,context)
 File "/py-test/python/lib/python2.7/urllib2.py", line 154, in urlopen
   return opener.open(url, data, timeout)
 File "/py-test/python/lib/python2.7/urllib2.py", line 429, in open
   response = self._open(req, data)
 File "/py-test/python/lib/python2.7/urllib2.py", line 447, in _open
   '_open', req)
 File "/py-test/python/lib/python2.7/urllib2.py", line 407, in _call_chain
   result = func(*args)
 File "/py-test/python/lib/python2.7/urllib2.py", line 1241, in https_open
   context=self._context)
 File "/py-test/python/lib/python2.7/urllib2.py", line 1198, in do_open
   raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>

It is definitely the hostname verification. If we use the correct certificate and correct hostname the request successful.

If we use the wrong certificate it fails with the following exception.

File "./test.py", line 8, in <module>
   context.load_verify_locations("/py-test/python/bin/certificate_bad.pem")
ssl.SSLError: [X509] PEM lib (_ssl.c:3027)

Therefore, we need help to understand how to configure Python to ignore the hostname verification.

One more question (can be asked in the separate thread).

Do we have in Python a trustore file that include all known CA? Like cacerts.jks in Java. Where can we find the trustore?

Added

We “want to verify that the certificate was signed by a valid CA”, but we “don't care whether it identifies the site you're actually connecting to”.

We need help to configure Python to ignore the hostname verification? What is mistake in our code? We have tried to create the code according to the documentation https://docs.python.org/2/library/ssl.html

Added 2

We have invested a lot of time but unfortunately we do not have the progress.

Is anyone has the working example in Python 2.7? I mean is the code works if you access with other URL then appears in a certificate. May be Python 2.7 cannot be configured to ignore the hostname verification?

What can be our problem? We use it on CentOS 6. May be it is related to OpenSSL? We use the latest version openssl-1.0.1e-57.el6.x86_64. May be we should upgrade to Python 3.x?

Answer 1

As you discovered, you can accomplish this by customizing the SSLContext object used for verifying the connection.

However, to hook this into urllib2.urlopen you'll need to build a custom opener and install that.

Here's an example:

import httplib
import socket
import ssl
import urllib2

import certifi


class InterceptedHttpsConnection(httplib.HTTPSConnection):
    def connect(self):
        # Open an unencrypted TCP socket first
        sock = socket.create_connection((self.host, self.port), self.timeout)

        # Build up a custom SSLContext. (Might be better to do this once rather
        # than on every request.)
        context = ssl.SSLContext(ssl.PROTOCOL_TLS)

        # We require the SSL context to verify the certificate.
        context.verify_mode = ssl.CERT_REQUIRED

        # But we instruct the SSL context to *not* validate the hostname.
        context.check_hostname = False

        # Load CA certificates from the certifi bundle.
        context.load_verify_locations(cafile=certifi.where())

        # Use our SSLContext object to wrap the bare socket into an SSL socket.
        self.sock = context.wrap_socket(sock, server_hostname=self.host)


class InterceptedHttpsHandler(urllib2.HTTPSHandler):
    def https_open(self, req):
        return self.do_open(InterceptedHttpsConnection, req)


def main():
    opener = urllib2.build_opener(InterceptedHttpsHandler)
    urllib2.install_opener(opener)

    contents = urllib2.urlopen('https://example.com/').read()
    print contents