Complete set of punctuation marks for Python (not just ASCII)

Question

Is there a listing or library that has all punctuations that we might commonly come across?

Normally I use string.punctuation, but some punctuation characters are not included in it, for example:

>>> "'" in string.punctuation True >>> "’" in string.punctuation False 

Answer 1

You might do better with this check:

>>> import unicodedata >>> unicodedata.category("'").startswith("P") True >>> unicodedata.category("’").startswith("P") True 

The Unicode categories P* are specifically for Punctuation:

connector (Pc), dash (Pd), initial quote (Pi), final quote (Pf), open (Ps), close (Pe), other (Po)

To prepare the exhaustive collection, which you can subsequently use for fast membership checks, use a set comprehension:

>>> import sys >>> from unicodedata import category >>> codepoints = range(sys.maxunicode + 1) >>> punctuation = {c for i in codepoints if category(c := chr(i)).startswith("P")} >>> "'" in punctuation True >>> "’" in punctuation True 

Assignment expression here requires Python 3.8+, equivalent for older Python versions:

chrs = (chr(i) for i in range(sys.maxunicode + 1)) punctuation = set(c for c in chrs if category(c).startswith("P")) 

Beware that some of the other characters in string.punctuation are actually in Unicode category Symbol. It's easy to add those in also if you want.

Answer 2

The answer posted by wim is correct if you want to check if a character is a punctuation character.

If you really need a list of all punctuation characters as your question title suggests, you can use the following:

import sys from unicodedata import category punctuation_chars =  [chr(i) for i in range(sys.maxunicode)                               if category(chr(i)).startswith("P")]