Python 3 urllib produces TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str

Question

I am trying to convert working Python 2.7 code into Python 3 code and I am receiving a type error from the urllib request module.

I used the inbuilt 2to3 Python tool to convert the below working urllib and urllib2 Python 2.7 code:

import urllib2 import urllib  url = "https://www.customdomain.com" d = dict(parameter1="value1", parameter2="value2")  req = urllib2.Request(url, data=urllib.urlencode(d)) f = urllib2.urlopen(req) resp = f.read() 

The output from the 2to3 module was the below Python 3 code:

import urllib.request, urllib.error, urllib.parse  url = "https://www.customdomain.com" d = dict(parameter1="value1", parameter2="value2")  req = urllib.request.Request(url, data=urllib.parse.urlencode(d)) f = urllib.request.urlopen(req) resp = f.read() 

When the Python 3 code is run the following error is produced:

--------------------------------------------------------------------------- TypeError                                 Traceback (most recent call last) <ipython-input-56-206954140899> in <module>()       5        6 req = urllib.request.Request(url, data=urllib.parse.urlencode(d)) ----> 7 f = urllib.request.urlopen(req)       8 resp = f.read()  C:\Users\Admin\Anaconda3\lib\urllib\request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)     159     else:     160         opener = _opener --> 161     return opener.open(url, data, timeout)     162      163 def install_opener(opener):  C:\Users\Admin\Anaconda3\lib\urllib\request.py in open(self, fullurl, data, timeout)     459         for processor in self.process_request.get(protocol, []):     460             meth = getattr(processor, meth_name) --> 461             req = meth(req)     462      463         response = self._open(req, data)  C:\Users\Admin\Anaconda3\lib\urllib\request.py in do_request_(self, request)    1110                 msg = "POST data should be bytes or an iterable of bytes. " \    1111                       "It cannot be of type str." -> 1112                 raise TypeError(msg)    1113             if not request.has_header('Content-type'):    1114                 request.add_unredirected_header(  TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str. 

I have also read two other tickets (ticket1 and ticket2) which mentioned encoding the date.

When I changed the line f = urllib.request.urlopen(req) to f = urllib.request.urlopen(req.encode('utf-8')) I received the following error: AttributeError: 'Request' object has no attribute 'encode'

I am stuck as to how to make the Python 3 code work. Could you please help me?

Answer 1

From the docs Note that params output from urlencode is encoded to bytes before it is sent to urlopen as data:

data = urllib.parse.urlencode(d).encode("utf-8") req = urllib.request.Request(url) with urllib.request.urlopen(req,data=data) as f:     resp = f.read()     print(resp) 

Answer 2

Try this:

url = 'https://www.customdomain.com' d = dict(parameter1="value1", parameter2="value2")  f = urllib.parse.urlencode(d) f = f.encode('utf-8')  req = urllib.request.Request(url, f) 

Your problem lies in the way you were handling the dictionary.