Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Correct way to define Python source code encoding

People also ask

How do you define encoding in Python?

Since Python 3.0, strings are stored as Unicode, i.e. each character in the string is represented by a code point. So, each string is just a sequence of Unicode code points. For efficient storage of these strings, the sequence of code points is converted into a set of bytes. The process is known as encoding.

What encoding should I use Python?

UTF-8 is one of the most commonly used encodings, and Python often defaults to using it. UTF stands for “Unicode Transformation Format”, and the '8' means that 8-bit values are used in the encoding. (There are also UTF-16 and UTF-32 encodings, but they are less frequently used than UTF-8.)

What is source code encoding?

Scheme source code files are usually encoded in ASCII or UTF-8, but the built-in reader can interpret other character encodings as well. When Guile loads Scheme source code, it uses the file-encoding procedure (described below) to try to guess the encoding of the file. In the absence of any hints, UTF-8 is assumed.


Check the docs here:

"If a comment in the first or second line of the Python script matches the regular expression coding[=:]\s*([-\w.]+), this comment is processed as an encoding declaration"

"The recommended forms of this expression are

# -*- coding: <encoding-name> -*-

which is recognized also by GNU Emacs, and

# vim:fileencoding=<encoding-name>

which is recognized by Bram Moolenaar’s VIM."

So, you can put pretty much anything before the "coding" part, but stick to "coding" (with no prefix) if you want to be 100% python-docs-recommendation-compatible.

More specifically, you need to use whatever is recognized by Python and the specific editing software you use (if it needs/accepts anything at all). E.g. the coding form is recognized (out of the box) by GNU Emacs but not Vim (yes, without a universal agreement, it's essentially a turf war).


Just copy paste below statement on the top of your program.It will solve character encoding problems

#!/usr/bin/env python
# -*- coding: utf-8 -*-

PEP 263:

the first or second line must match the regular expression "coding[:=]\s*([-\w.]+)"

So, "encoding: UTF-8" matches.

PEP provides some examples:

#!/usr/bin/python
# vim: set fileencoding=<encoding name> :

 

# This Python file uses the following encoding: utf-8
import os, sys

As of today — June 2018


PEP 263 itself mentions the regex it follows:

To define a source code encoding, a magic comment must be placed into the source files either as first or second line in the file, such as:

# coding=<encoding name>

or (using formats recognized by popular editors):

#!/usr/bin/python
# -*- coding: <encoding name> -*-

or:

#!/usr/bin/python
# vim: set fileencoding=<encoding name> : 

More precisely, the first or second line must match the following regular expression:

^[ \t\f]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)

So, as already summed up by other answers, it'll match coding with any prefix, but if you'd like to be as PEP-compliant as it gets (even though, as far as I can tell, using encoding instead of coding does not violate PEP 263 in any way) — stick with 'plain' coding, with no prefixes.


If I'm not mistaken, the original proposal for source file encodings was to use a regular expression for the first couple of lines, which would allow both.

I think the regex was something along the lines of coding: followed by something.

I found this: http://www.python.org/dev/peps/pep-0263/ Which is the original proposal, but I can't seem to find the final spec stating exactly what they did.

I've certainly used encoding: to great effect, so obviously that works.

Try changing to something completely different, like duhcoding: ... to see if that works just as well.