Flask SecureCookie replacing pickle with json results in encoding error

Question

Below is werkzeug's recommendation for session serialization:

The default implementation uses Pickle as this is the only module that used to be available in the standard library when this module was created. If you have simplejson available it’s strongly recommended to create a subclass and replace the serialization method:

When I do this following both the Flask implementation below:

https://gist.github.com/runfalk/2501926

or alternatively

from werkzeug.contrib.securecookie import SecureCookie
Secure_Cookie.serialization_method = json

UnicodeDecodeError: 'utf8' codec can't decode byte 0x97 in position 0: invalid start byte

This happens when I attempt to login.

I have delved into the json encoder source and attempted to set ensure_ascii=False and this got me past the error above but then I just couldn't log into the application at all. It would just flash the screen with no error and I remained stuck at the login prompt. With pickle everything works fine.

An important side note this problem is unique to the integration of the Flask-login and does not occur with just the vanilla flask session serialization

ASKSBADQUESTIONS's code does indeed work but this throws the decode error

import json

from flask import Flask, session
from flask.sessions import SecureCookieSession, SecureCookieSessionInterface
from flask.ext.login import LoginManager

class JSONSecureCookieSession(SecureCookieSession):
    serialization_method = json

class JSONSecureCookieSessionInterface(SecureCookieSessionInterface):
    session_class = JSONSecureCookieSession

app = Flask(__name__)
app.secret_key = "I-like-cookies-and-some-secure-cookies"
app.session_interface = JSONSecureCookieSessionInterface()

#Initialize Login Manager
login_manager = LoginManager()
login_manager.setup_app(app)

@app.route("/")
def hello():
    k = "lalala"

    v = session.get(k)

    if v is None:
        print "set"
        v = session[k] = "FLAAASK abuses decorators in a bad way :)"
    else:
        print "get"

    return "Hello {0}".format(v)


if __name__ == "__main__":
    app.run(debug=True)

And here is the stacktrace

Traceback (most recent call last):
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/flask/app.py", line 1689, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/flask/app.py", line 1362, in full_dispatch_request
    response = self.process_response(response)
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/flask/app.py", line 1566, in process_response
    self.save_session(ctx.session, response)
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/flask/app.py", line 804, in save_session
    return self.session_interface.save_session(self, session, response)
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/flask/sessions.py", line 205, in save_session
    secure=secure, domain=domain)
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/werkzeug/contrib/securecookie.py", line 329, in save_cookie
    data = self.serialize(session_expires or expires)
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/werkzeug/contrib/securecookie.py", line 235, in serialize
    self.quote(value)
  File "/usr/local/pythonbrew/venvs/Python-2.7.3/flask-session-bug/lib/python2.7/site-packages/werkzeug/contrib/securecookie.py", line 192, in quote
    value = cls.serialization_method.dumps(value)
  File "/usr/local/pythonbrew/pythons/Python-2.7.3/lib/python2.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/pythonbrew/pythons/Python-2.7.3/lib/python2.7/json/encoder.py", line 195, in encode
    return encode_basestring_ascii(o)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x97 in position 0: invalid start byte
set

Answer 1

There is a method in flask_login.py called _create_identifier. The result of this invocation is a md5 digest() result. According to the docs this can contain non-ascii characters and/or null bytes. When handed to the serialization_method it is unable to decode the unicode bytes.

This bug definitely exists in the combination of Flask 0.9 and Flask-login 0.1.3 and can be fixed by monkey patching this gist(https://gist.github.com/anonymous/3731115) into the flask_login.py file or you can pull the latest out of development.

You can refer to this bug on their github repo if you need more info https://github.com/maxcountryman/flask-login/pull/31

Answer 2

I quickly wrote minimal Flask app to reproduce your bug (saved into app.py; invoked with python app.py):

import json

from flask import Flask, session
from flask.sessions import SecureCookieSession, SecureCookieSessionInterface


class JSONSecureCookieSession(SecureCookieSession):
    serialization_method = json


class JSONSecureCookieSessionInterface(SecureCookieSessionInterface):
    session_class = JSONSecureCookieSession


app = Flask(__name__)
app.secret_key = "I-like-cookies-and-some-secure-cookies"
app.session_interface = JSONSecureCookieSessionInterface()


@app.route("/")
def hello():
    k = "lalala"

    v = session.get(k)

    if v is None:
        print "set"
        v = session[k] = "FLAAASK abuses decorators in a bad way :)"
    else:
        print "get"

    return "Hello {0}".format(v)


if __name__ == "__main__":
    app.run(debug=True)

But everything went smoothly. I even changed session_interface from Pickle-based to JSON-based few times, and did not get exception. Perhaps you should post full exception traceback and (maybe) add some code to my example to reproduce a bug.