Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Match letter in any language

Tags:

How can I match a letter from any language using a regex in python 3?

re.match([a-zA-Z]) will match the english language characters but I want all languages to be supported simultaneously.

I don't wish to match the ' in can't or underscores or any other type of formatting. I do wish my regex to match: c, a, n, t, Å, é, and .

like image 386
Baz Avatar asked Aug 26 '11 14:08

Baz


1 Answers

For Unicode regex work in Python, I very strongly recommend the following:

  1. Use Matthew Barnett’s regex library instead of standard re, which is not really suitable for Unicode regular expressions.
  2. Use only Python 3, never Python 2. You want all your strings to be Unicode strings.
  3. Use only string literals with logical/abstract Unicode codepoints, not encoded byte strings.
  4. Set your encoding on your streams and forget about it. If you find yourself ever manually calling .encode and such, you’re almost certainly doing something wrong.
  5. Use only a wide build where code points and code units are the same, never ever ever a narrow one — which you might do well to consider deprecated for Unicode robustness.
  6. Normalize all incoming strings to NFD on the way in and then NFC on the way out. Otherwise you can’t get reliable behavior.

Once you do this, you can safely write patterns that include \w or \p{script=Latin} or \p{alpha} and \p{lower} etc and know that these will all do what the Unicode Standard says they should. I explain all of this business of Python Unicode regex business in much more detail in this answer. The short story is to always use regex not re.

For general Unicode advice, I also have several talks from last OSCON about Unicode regular expressions, most of which apart from the 3rd talk alone is not about Python, but much of which is adaptable.

Finally, there’s always this answer to put the fear of God (or at least, of Unicode) in your heart.

like image 192
tchrist Avatar answered Sep 19 '22 16:09

tchrist