How to fix InsecureRequestWarning: Unverified HTTPS request is being made to host

Question

I am trying to download a zip file from url, but I get the below warning

InsecureRequestWarning: Unverified HTTPS request is being made to host 'www.ons.gov.uk'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings   

I read about this a bit and most of the threads revolves around how to disable it (if you know what you are doing) and the general concept as a whole. From what I understood, isn't request library capable of performing certificate validation by default? Why should I make a secured request to a open source file? I am exposing myself to something here, and how can I fix this ?

Here's my code

import pandas as pd
import requests
from requests.auth import HTTPBasicAuth
from zipfile import ZipFile
import io

url = "https://www.ons.gov.uk/file?uri=%2fpeoplepopulationandcommunity%2fhousing%2fdatasets%2fukhousebuildingpermanentdwellingsstartedandcompleted%2fcurrent/ukhousebuilding.zip"


response = requests.get(url, headers={'User-Agent': 'Mozilla/5.0'}, verify=False)
with ZipFile(io.BytesIO(response.content)) as myzip:
    with myzip.open(myzip.namelist()[0]) as hfile:
        df = pd.read_csv(hfile)

print(df)

Apparently this warning only turns up when I use verify=False, shouldn't it actually prevent this warning?

Sorry for asking too many questions, but I am trying to understand what's happening here. Please correct me if I have misunderstood the concept.

Answer 1

From what I understood, isn't request library capable of performing certificate validation by default?

Yes, it does.

Why should I make a secured request to a open source file?

For security reasons. You want to be sure you actually get the file you expect, from the source you request it from without any manipulations to the file.

I am exposing myself to something here, and how can I fix this?

Yes, you are. Remove the verify=False parameter.

Apparently this warning only turns up when I use verify=False, shouldn't it actually prevent this warning?

No, it's exactly what causes the warning. Read it carefully. It warns you about unverified HTTPS requests. The request is unverified because you specified it.

TL;DR

If the requests works without the verify=False parameter, you should not use it. Otherwise you should still not use it and find another solution.