Python3 error: initial_value must be str or None, with StringIO

Question

While porting code from python2 to 3, I get this error when reading from a URL

TypeError: initial_value must be str or None, not bytes.

import urllib
import json
import gzip
from urllib.parse import urlencode
from urllib.request import Request


service_url = 'https://babelfy.io/v1/disambiguate'
text = 'BabelNet is both a multilingual encyclopedic dictionary and a semantic network'
lang = 'EN'
Key  = 'KEY'

    params = {
        'text' : text,
        'key'  : Key,
        'lang' :'EN'

        }

url = service_url + '?' + urllib.urlencode(params)
request = Request(url)
request.add_header('Accept-encoding', 'gzip')
response = urllib.request.urlopen(request)
if response.info().get('Content-Encoding') == 'gzip':
            buf = StringIO(response.read())
            f = gzip.GzipFile(fileobj=buf)
            data = json.loads(f.read())

The exception is thrown at this line

buf = StringIO(response.read())  

If I use python2, it works fine.

Answer 1

response.read() returns an instance of bytes while StringIO is an in-memory stream for text only. Use BytesIO instead.

From What's new in Python 3.0 - Text Vs. Data Instead Of Unicode Vs. 8-bit

The StringIO and cStringIO modules are gone. Instead, import the io module and use io.StringIO or io.BytesIO for text and data respectively.

Answer 2

This looks like another python3 bytes vs. str problem. Your response is of type bytes (which is different in python 3 from str). You need to get it into a string first using response.read().decode('utf-8') say and then use StringIO on it. Or you may want to use BytesIO as someone said - but if you expect it to be str, preferred way is to decode into an str first.

Answer 3

Consider using six.StringIO instead of io.StringIO.