Python3 wrong string encoding

Question

I've got an application where you can login via SAML2. I'm using apache mellon module and getting data:

name = request.environ['MELLON_name']
email = request.environ['MELLON_mail']

From those data I create JWT using flask_jwt_simpe library. Then I want to call get_jwt_identity(), but the name in response has wrong encoding, it looks JiÅí Manes instead of Jiří Manes (Czech language). How can i solve this problem?

Edit #1: locale command output

LANG=en_US.utf8
LANGUAGE=
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=en_US.utf8

Edit #2: Solved it on my VPS by following python code:

name = bytearray(request.environ['MELLON_name'], 'iso-8859-1').decode('utf-8')

But I would like to have another universal solution :-/

Answer 1

You have hit the WSGI encoding dance. Unfortunately, there isn't really a better solution than the one you've already found.

As you can see, you are already doing the same as werkzeug (which is the WSGI package used by Flask) to solve this issue.

If you wanted, you could use the compatibility functions in that package, but you may find they change without notification being in a private module, so you're probably best off sticking to your own equivalent code.