Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Javascript unicode (greek) regular expressions

I would like to use this regular expression new RegExp("\b"+pat+"\b") in greek text but the "\b" metacharacter supports only ASCII characters.

I tried XregExp library but i didnt manage to solve the issue.

Any suggestions would be greatly appreciated.

like image 914
kylito Avatar asked Apr 13 '11 13:04

kylito


People also ask

Does JavaScript regex support Unicode?

JavaScript, which does not offer any Unicode support through its RegExp class, does support \uFFFF for matching a single Unicode code point as part of its string syntax. XML Schema and XPath do not have a regex token for matching Unicode code points.

What is \p l in regex?

\p{L} matches a single code point in the category "letter". \p{N} matches any kind of numeric character in any script. Source: regular-expressions.info.

What is the regex for Unicode paragraph seperator?

\u000d — Carriage return — \r. \u2028 — Line separator. \u2029 — Paragraph separator.

What is U flag in regex?

Flag u enables the support of Unicode in regular expressions. That means two things: Characters of 4 bytes are handled correctly: as a single character, not two 2-byte characters. Unicode properties can be used in the search: \p{…} .


2 Answers

I think this was helpful to your answer.,

<script src="xregexp.js"></script>
<script src="xregexp-unicode-base.js"></script>
<script>
    var unicodeWord = XRegExp("^\\p{L}+$");

    unicodeWord.test("Русский"); // true
    unicodeWord.test("日本語"); // true
    unicodeWord.test("العربية"); // true
</script>

<!-- \p{L} is included in the base script, but other categories, scripts,
and blocks require token packages -->
<script src="xregexp-unicode-scripts.js"></script>
<script>
    XRegExp("^\\p{Katakana}+$").test("カタカナ"); // true
</script>

Please refer the following location : http://xregexp.com/plugins/

like image 114
John Peter Avatar answered Oct 12 '22 23:10

John Peter


So the answer is just, that you can not use the JavaScript native mechanisms or any library which uses those mechanisms to match words the way you want to. As you already stated, \b matches words. Words must consists of word characters. And in JavaScript (and actually other regex implementations word characters are a-z, A-Z, 0-9 and _. But many other Languages just implement the \b metacharacter in a different way JavaScript does.

The answer "JavaScript does not support Unicode" is a bit to easy and in fact completely wrong. JavaScript just doesn't use unicode for the character classes. If JavaScript wouldn't support unicode you couldn't even use unicode Characters in String literals and of course this is possible in JavaScript.

According to the ECMA 262 Standard (ECMAScript) (Section 15.10.2.6):

[...] The production Assertion :: \ b evaluates by returning an internal AssertionTester closure that takes a State argument x and performs the following:

  1. Let e be x's endIndex.
  2. Call IsWordChar(e–1) and let a be the Boolean result.
  3. Call IsWordChar(e) and let b be the Boolean result.
  4. If a is true and b is false, return true.
  5. If a is false and b is true, return true.
  6. Return false. [..]

The abstract operation IsWordChar takes an integer parameter e and performs the following:

  1. If e == –1 or e == InputLength, return false.
  2. Let c be the character Input[e].
  3. If c is one of the sixty-three characters below, return true. a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 0 1 2 3 4 5 6 7 8 9 _
  4. Return false

This just shows, that the \b uses the Algorithm of "isWordChar" to check if what you try to match is actually a word. Int he definition of "isWordChar" you can see the exact definition of which characters will return true for "isWordChar".

In my Opinion this has absolutely nothing to do with the character set being used. It's neither ASCII nor UNICODE compilant here. It's just these 63 characters.

like image 24
Chris Avatar answered Oct 13 '22 00:10

Chris