Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Split string into sentences - ignoring abbreviations for splitting

I'm trying to split this string into sentences, but I need to handle abbreviations (which have the fixed format x.y. as a word:

content = "This is a long string with some numbers 123.456,78 or 100.000 and e.g. some abbreviations in it, which shouldn't split the sentence. Sometimes there are problems, i.e. in this one. here and abbr at the end x.y.. cool."

I tried this regex:

content.replace(/([.?!])\s+(?=[A-Za-z])/g, "$1|").split("|");

But as you can see there are problems with abbreviations. As all the abbreviations are of the format x.y. it should be possible to handle them as a word, without splitting the string at this point.

"This is a long string with some numbers 123.456,78 or 100.000 and e.g.", 
"some abbreviations in it, which shouldn't split the sentence."
"Sometimes there are problems, i.e.", 
"in this one.", 
"here and abbr at the end x.y..",
"cool."

The result should be:

"This is a long string with some numbers 123.456,78 or 100.000 and e.g. some abbreviations in it, which shouldn't split the sentence."
"Sometimes there are problems, i.e. in this one.", 
"here and abbr at the end x.y..",
"cool."
like image 953
user3142695 Avatar asked Jan 14 '16 08:01

user3142695


2 Answers

The solution is to match and capture the abbreviations and build the replacement using a callback:

var re = /\b(\w\.\w\.)|([.?!])\s+(?=[A-Za-z])/g; 
var str = 'This is a long string with some numbers 123.456,78 or 100.000 and e.g. some abbreviations in it, which shouldn\'t split the sentence. Sometimes there are problems, i.e. in this one. here and abbr at the end x.y.. cool.';
var result = str.replace(re, function(m, g1, g2){
  return g1 ? g1 : g2+"\r";
});
var arr = result.split("\r");
document.body.innerHTML = "<pre>" + JSON.stringify(arr, 0, 4) + "</pre>";

Regex explanation:

  • \b(\w\.\w\.) - match and capture into Group 1 the abbreviation (consisting of a word character, then . and again a word character and a .) as a whole word
  • | - or...
  • ([.?!])\s+(?=[A-Za-z]):
    • ([.?!]) - match and capture into Group 2 either . or ? or !
    • \s+ - match 1 or more whitespace symbols...
    • (?=[A-Za-z]) - that are before an ASCII letter.
like image 72
Wiktor Stribiżew Avatar answered Sep 30 '22 05:09

Wiktor Stribiżew


Given your example, I have managed to achieve what you are after through the use of this expression: (?<!\..)[.?!]\s+ (example here).

This expression will look for period, question mark or exclamation mark characters which are not preceded by a character and a period.

You would then need to replace them with the | character and finally, you replace the | with .\n.

like image 41
npinti Avatar answered Sep 30 '22 06:09

npinti