Regular expression with the cyrillic alphabet

Question

I have an jQuery function for word counting in textarea field. In addition its excludes all words, which are closed in [[[tripple bracket]]]. It works great with latin character, but it has a problem with cyrillic sentences. I suppose that the error is in part with regular expression:

$(field).val().replace(/\[\[\[[^\]]*\]\]\]/g, '').match(/\b/g); 

Example with both kind of phrases: http://jsfiddle.net/A3cEG/2/

I need count all word, including cirillic expressions, not only words in latin. How to do that?

Answer 1

JavaScript (at least the versions most widely used) does not fully support Unicode. That is to say, \w matches only Latin letters, decimal digits, and underscores ([a-zA-Z0-9_]), and \b matches the boundary the between a word character and and a non-word character.

To find all words in an input string using Latin or Cyrillic, you'd have to do something like this:

.match(/[\wа-я]+/ig); // where а is the Cyrillic а. 

Or if you prefer:

.match(/[\w\u0430-\u044f]+/ig); 

Of course this will probably mean you need to tweak your code a little bit, since here it will match all words rather than word boundaries. Note that [а-я] matches any letter in the 'basic Cyrillic alphabet' as described here. To match letters outside of this range, you can modify the character set as necessary to include those letters, e.g. to also match the Russian Ё/ё, use [а-яё].

Also note that your triple-bracket pattern can be simplified to:

.replace(/\[{3}[^]]*]{3}/g, '') 

Alternatively, you might want to look at the XRegExp project—which is an open-source project to add new features to the base JavaScript regular expression engine—and its Unicode addon.

Answer 2

Beware of using range of cyrillic letters, it may contain unnecessary characters within. There is bulletproof regexp contains only cyrillic letters:

/^[аАбБвВгГдДеЕёЁжЖзЗиИйЙкКлЛмМнНоОпПрРсСтТуУфФхХцЦчЧшШщЩъЪыЫьЬэЭюЮяЯ]+$/