Docker Python set utf-8 locale

Question

I am trying to run my python file that first reads a string in Chinese language and print it.

This is my Dockerfile

FROM python:2.7-onbuild
ENV LANG en_US.UTF-8
ADD . /code
WORKDIR /code
RUN pip install -r requirements.txt
CMD ["python", "app.py"]

This is my python file:

    # -*- coding: utf-8 -*-
    import jieba
    s = "我来到北京清华大学"
    s = s.decode('utf-8')
    print type(s), s

I then run :

docker build -t python-example .

docker run python-example

Error i got: UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-8: ordinal not in range(128)

When i run it locally, it works fine.

Answer 1

Short version

Put this in your Dockerfile:

ENV PYTHONIOENCODING=utf-8

or as mentioned in the comments above pass it on the command line:

docker run -e PYTHONIOENCODING=utf-8 my-python-image some-command

Long version:

When you start the Python interpreter, Python has to set up stdout in order to send output to your terminal. On your modern O/S, your terminal probably reports that it supports UTF-8 or some other advanced encoding. You can see what encoding is used by running this command:

$ python -c 'import sys; print(sys.stdout.encoding)'
UTF-8

When you run a docker container, the environment variables Python would expect to use a more advanced encoding are not present, and so Python will fall back to a basic character set to ensure compatibility. You can verify this by running the same command in your container:

$ docker run my-python-image python -c 'import sys; print(sys.stdout.encoding)'
ANSI_X3.4-1968

When we pass PYTHONIOENCODING we see the sys.stdout.encoding is set appropriately:

$ docker run -e PYTHONIOENCODING=UTF-8 my-python-image python -c 'import sys; print(sys.stdout.encoding)'
UTF-8

Read about PYTHONIOENCODING in the Python documentation. This answer also goes into great detail about encoding/decoding and stdout.

Answer 2

I ran into the same issue while I was deploying a Django application with supervisor and gunicorn.

What fixed it was to add the following line to my supervisor config file:

environment=LANG="es_ES.utf8", LC_ALL="es_ES.UTF-8", LC_LANG="es_ES.UTF-8"

For your case make sure that the chinese locale that you want to print is available and installed in your docker container. This blog describes how to do it: example dockerfile (use the chinese locale instead of en_CA.UTF-8):

FROM ubuntu:15.10
MAINTAINER Mobify <[email protected]>

RUN apt-get -qq update && \
    apt-get -q -y upgrade && \
    apt-get install -y sudo curl wget locales && \
    rm -rf /var/lib/apt/lists/*

# Ensure that we always use UTF-8 and with Canadian English locale
RUN locale-gen en_CA.UTF-8

COPY ./default_locale /etc/default/locale
RUN chmod 0755 /etc/default/locale

ENV LC_ALL=en_CA.UTF-8
ENV LANG=en_CA.UTF-8
ENV LANGUAGE=en_CA.UTF-8

hopefully this leads you into the right direction.

Answer 3

I add the below command in my docker file：

RUN locale-gen en_US.UTF-8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'

then build/rebuild docker images, you'd better add this in the base image.