Python error: Type error: POST data should be bytes; also user-agent issue

Question

Using the following code I received an error:

TypeError: POST data should be bytes or an iterable of bytes. It cannot be str

Second concern, I am not sure if I specified my user-agent correctly, here's my user-agent in whole: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4. I gave my best shot as I defined the user-agent in the script.

import urllib.parse
import urllib.request

url = 'http://getliberty.org/contact-us/'
user_agent = 'Mozilla/5.0 (compatible; Chrome/22.0.1229.94; Windows NT)'
values = {'Your Name' : 'Horatio',
          'Your Email' : '[email protected]',
          'Subject' : 'Hello',
          'Your Message' : 'Cheers'}

headers = {'User-Agent': user_agent }

data = urllib.parse.urlencode(values)
req = urllib.request.Request(url, data)
response = urllib.request.urlopen(req)
the_page = response.read()

I am aware of this similar question, TypeError: POST data should be bytes or an iterable of bytes. It cannot be str, but am too new for the answer to be much help.

Answer 1

data = urllib.parse.urlencode(values)
type(data) #this returns <class 'str'>. it's a string

The urllib docs say for urllib.request.Request(url, data ...):

The urllib.parse.urlencode() function takes a mapping or sequence of 2-tuples and returns a string in this format. It should be encoded to bytes before being used as the data parameter. etc etc

(emphasis mine)

So you have a string that looks right, what you need is that string encoded into bytes. And you choose the encoding.

binary_data = data.encode(encoding)

in the above line: encoding can be 'utf-8' or 'ascii' or a bunch of other things. Pick whichever one the server expects.

So you end up with something that looks like:

data = urllib.parse.urlencode(values)
binary_data = data.encode(encoding) 
req = urllib.request.Request(url, binary_data)

Answer 2

You can try with requests module as an alternative solution

import json
import requests

url = 'http://getliberty.org/contact-us/'
user_agent = 'Mozilla/5.0 (compatible; Chrome/22.0.1229.94; Windows NT)'
values = {
      'Your Name' : 'Horatio',
      'Your Email' : '[email protected]',
      'Subject' : 'Hello',
      'Your Message' : 'Cheers'
       }

headers = {'User-Agent': user_agent, 'Content-Type':'application/json' }

data = json.dumps(values)
request = requests.post(url, data=data, headers=headers)

response = request.json()