Object of type 'bytes' is not JSON serializable when upgrading my python environment

Question

I have developed an application in Flask that worked. I try to upgrade my environment from version 2.7 to 3.6. When I am hitting an endpoint that is calling a form It generated the following error:

Traceback (most recent call last):
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/TheSwitchWebApp/controllers/account/register.py", line 15, in display_signup_form
    form = RegisterForm(request.form)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/wtforms/form.py", line 212, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask_wtf/form.py", line 88, in __init__
    super(FlaskForm, self).__init__(formdata=formdata, **kwargs)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/wtforms/form.py", line 278, in __init__
    self.process(formdata, obj, data=data, **kwargs)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/wtforms/form.py", line 132, in process
    field.process(formdata)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/wtforms/csrf/core.py", line 43, in process
    self.current_token = self.csrf_impl.generate_csrf_token(self)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask_wtf/csrf.py", line 134, in generate_csrf_token
    token_key=self.meta.csrf_field_name
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/flask_wtf/csrf.py", line 47, in generate_csrf
    setattr(g, field_name, s.dumps(session[field_name]))
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/itsdangerous.py", line 565, in dumps
    payload = want_bytes(self.dump_payload(obj))
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/itsdangerous.py", line 847, in dump_payload
    json = super(URLSafeSerializerMixin, self).dump_payload(obj)
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/itsdangerous.py", line 550, in dump_payload
    return want_bytes(self.serializer.dumps(obj))
  File "/Users/Melvyn/Documents/personnel/python/TheSwitchWebApp/venv/lib/python3.6/site-packages/itsdangerous.py", line 51, in dumps
    return json.dumps(obj, separators=(',', ':'))
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'bytes' is not JSON serializable

It generate this error when I'm hitting the following endpoint:

@register_endpoint.route('/', methods=['GET'])
def display_signup_form():
    error = None
    form = RegisterForm(request.form)
    return render_template("auth/register.html", form=form, error=error)

The error seem to come from the form:

class RegisterForm(Form):
    first_name = StringField('First name', [InputRequired(message='First name is mandatory')])
    email = StringField('Email Address', [Email(), InputRequired(message='Forgot your email address?')])

I don't get the source of the error. In this piece of code I do not try to serialise in JSON anything. Where does the error come from?

Answer 1

As @Arpit Solanki mentioned, py2.x and py3.x are very different in case of encoding. You'd better clear your cookies in your browser, probably the browser had cookies set by python2 code, but your new python3 interpreted the cookie content as byte strings. Please clear your cookie will solve the issue.

You can also selectively wipe just the offending cookie.