Can regular expressions find repetitions of characters?

Question

My users insert sequences like

________________________
************************
------------------------
♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥

to format documents (dont ask me about my users!). And it looks bad when displaying snippets. How can I remove repetitions of any characters? I can add individual filters, but it will be a constant cat and mouse game.

Can a regular expression filter these?

Answer 1

Try something like:

(.)\1{5,}

Which matches any character, then 5 or more of that character. Remember to escape the \ if your language uses strings for regex patterns!

Answer 2

You can remove repetitions of any character with a simple regex like (.)\1+

However, this will catch legitimate uses as well, such as words that have doubled letters in their spelling (balloon, spelling, well, etc).

So, you'd probably want to restrict the expression to some disallowed characters, after all, while keeping it as generic as possible, in order not to have to modify it from time to time, as your users find new characters to use.
One possible solution would be to disallow repeated non-letter and non-number characters:

([^A-Za-z0-9])\1+

But even this is not a definitive solution to all the cases, as some of your users may actually decide to use actual letter sequences as delimiters:

ZZZZZZZZZZZZZZZZZZZZZZ
BBBBBBBBBBBBBBBBBBBBBB
ZZZZZZZZZZZZZZZZZZZZZZ

In order not to allow this and with the added benefit of allowing legitimate uses of some repeated non-letter characters (such as in an ellipsis: ...), you could restrict the character repetitions to a maximum of 3, by using a regex with the syntax (<pattern>)\1{min, max} like this: (.)\1{4,} to match offending character sequences, with a minimum length of 4 and an unspecified maximum.

Answer 3

In python (but the logic is the same regardless of the language):

>>> import re
>>> text = '''
... This is some text
... ________________________
... This some more
... ♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥
... Truly the last line
... '''
>>> print re.sub(r'[_♥]{2,}', '', text)  #this is the core (regexp)

This is some text

This some more

Truly the last line

This has the advantage that you have some control on what to substitute and what not (for example you might wish not to substitute . as it could be part of a comment like This is still to do....

EDIT:

If your repetitions are always "lines" you could add the newline characters to your expression:

text = '''
This is some text
________________________
This some more
♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥♥
Truly the last line
But this is not to be changed: ♥♥♥
'''
>>> print re.sub(r'\n[_♥]{2,}\n', '\n', text)
This is some text
This some more
Truly the last line
But this is not to be changed: ♥♥♥

HTH