Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to implement \p{L} in python regex

I was trying to match all the string that contain one word in any language. My search led me to \p{...} which was absent in python's Re module. But I found https://pypi.python.org/pypi/regex. It should work with \p{...} commands. Although it doesn't.

I tried parsing those lines:

7652167371  apéritif
78687   attaché
78687   époque
78678   kunngjøre
78678   ærbødig
7687    vår
12312   dfsdf
23123   322432
1321    23123
2312    привер
32211   оипвыола

With:

def Pattern_compile(pattern_array):
    regexes = [regex.compile(p) for p in pattern_array]
    return regexes

def main():
    for line in sys.stdin:
        for regexp in Pattern_compile(p_a):
            if regexp.search (line):
                print line.strip('\n')

if __name__ == '__main__':
    p_a = ['^\d+\t(\p{L}|\p{M})+$', ]
    main()

The result is only latin-character word:

12312   dfsdf
like image 672
antonavy Avatar asked Oct 21 '22 05:10

antonavy


1 Answers

You should pass unicode. (Both regular expression and the string)

import sys

import regex


def main(patterns):
    patterns = [regex.compile(p) for p in patterns]
    for line in sys.stdin:
        line = line.decode('utf8')
        for regexp in patterns:
            if regexp.search (line):
                print line.strip('\n')

if __name__ == '__main__':
    main([ur'^\d+\t(\p{L}|\p{M})+$', ])
like image 144
falsetru Avatar answered Nov 01 '22 11:11

falsetru