urllib.error.URLError: <urlopen error unknown url type: 'https>

Question

(Python 3.4.2) I've got a weird error when I run 'urllib.request.urlopen(url)' inside of a script. If I run it directly in the Python interpreter, it works fine, but not when I run it inside of a script through a bash shell (Linux).

I'm guessing it has something to do with the 'url' string, maybe because I'm creating the string through the 'string.join' method.

import urllib.request
url = "".join((baseurl, other_string, midurl, query))
response = urllib.request.urlopen(url)

The 'url' string prints perfectly, but when I try to create the 'response' string, I get this output:

File "./script.py", line 124, in <module>
    response = urllib.request.urlopen(url)
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 455, in open
    response = self._open(req, data)
  File "/usr/lib/python3.4/urllib/request.py", line 478, in _open
    'unknown_open', req)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 1244, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: 'https>

Python was compiled with SSL support on my computer (these commands work perfectly in the Python interpreter).

I've also tried wrapping the 'url' string with 'repr(url)' and 'str(url)'. I've tried this too:

url = "".join(("'", baseurl, other_string, midurl, query, "'"))

Anyone know what's going on?

-----EDIT-----
I figured it out. My url had a ":" in it, and I guess urllib didn't like that. I replaced it with "%3A" and now it's working.

Answer 1

You should use urllib.parse.urlencode(), urllib.parse.urljoin(), etc functions to construct urls instead of manually joining the strings. It would take care of : -> %3A conversion e.g.:

>>> import urllib.parse
>>> urllib.parse.quote(':')
'%3A'

Answer 2

I figured it out. My url had a : in it, and urllib cannot use that character. I replaced it with %3A and now it's working. Web browsers usually convert : to %3A automatically, but urllib requires it to be converted first.

Answer 3

may due to openssl-devel if you do not install it.

yum list installed|grep openssl

install it and try again after make.

sudo yum install openssl-devel
./configure
make