Using range in regex for Arabic letters

Question

When using Regex in Python, it's easy to use brackets to represent a range of characters a-z, but this doesn't seem to be working for other languages, like Arabic:

import re
pattern = '[ي-ا]'
p = re.compile(pattern)

This results in a long error report that ends with

raise error("bad character range")
sre_constants.error: bad character range

how can this be fixed?

Answer 1

Since Arabic character is rendered from right to left, the correct string below, which reads "from ا to ي" is rendered backward (try to select the string if you want to confirm):

'[ا-ي]'

Console output:

>>> re.compile('[ا-ي]')
<_sre.SRE_Pattern object at 0x6001f0a80>

>>> re.compile('[ا-ي]', re.DEBUG)
in
  range (1575, 1610)
<_sre.SRE_Pattern object at 0x6001f0440>

So your pattern '[ي-ا]', is actually "from ي to ا", which is an invalid range, since the code point of ا is smaller than code point of ي.

To prevent confusion, Ignacio Vazquez-Abrams's suggestion of using Unicode escape is a good alternative to the solution I provide above.

Answer 2

Use Unicode escapes instead.

>>> re.compile('[\u0627-\u064a]')
<_sre.SRE_Pattern object at 0x237f460>