Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Split text on paragraphs where paragraph delimiters are non-standard

If I have text with standard paragraph formatting (a blank line followed by an indent) such as text 1 it's easy enough to extract the paragraphs using text.split("\n\n").

Text 1:

      Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus sit amet sapien velit, ac sodales   
 ante. Integer mattis eros non turpis interdum et auctor enim consectetur, etc.

      Praesent molestie suscipit bibendum. Donec justo purus, venenatis eget convallis sed, feugiat    
 vitae velit,etc.

But what if I have text with non-standard paragraph formatting such as text 2? No blank lines and variable leading whitespace.

Text 2:

      Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus sit amet sapien velit, ac sodales   
 ante. Integer mattis eros non turpis interdum et auctor enim consectetur, etc.
    Praesent molestie suscipit bibendum. Donec justo purus, venenatis eget convallis sed, feugiat    
 vitae velit,etc.

Since leading whitespace is common to both standard and non-standard formats I've thought about indexing on the regex match for leading whitespace and getting the paragraph breaks that way, but there has to be a more elegant way to do this.

like image 464
Renklauf Avatar asked Apr 10 '13 19:04

Renklauf


1 Answers

The regex solution you propose seems elegant enough:

re.split('\s{4,}',text)

This uses 4 consecutive whitespace chars as paragraph delimiter. You can use '\n\s{3,}' or something similar, if it fits better.

like image 92
Ofri Raviv Avatar answered Nov 15 '22 12:11

Ofri Raviv