Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache tika: remove extra line breaks in result string

I have html file:

<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;">
<div>Test message.</div>
<div>&nbsp;</div>
<div>More content here...</div>
<div>&nbsp;</div>
<div>Best regards,</div>
<div>Mr. Crowley</div></div></body></html>

I try to get content of the file above using Apache Tika...

final InputStream input = new FileInputStream("file.html");
final ContentHandler handler = new BodyContentHandler();
final Metadata metadata = new Metadata();

final HtmlParser htmlParser = new HtmlParser();
htmlParser.parse(input, handler, metadata, new ParseContext());
String plainText = handler.toString();
System.out.println(plainText);

...and all is fine except extra linebreaks:

Test message.

 

More content here...

 

Best regards,

Mr. Crowley
<and 3 empty lines here>

Is it possible to avoid this behavior? Is it possible to get more expected result:

Test message.
 
More content here...
 
Best regards,
Mr. Crowley

?

Code constructions like

plainText = plainText.replaceAll("(\n)+", "\n");

are unfortunately impossible here for me. Also I can't change the structure of my HTML file.

like image 431
hard-code Avatar asked Jul 04 '13 17:07

hard-code


1 Answers

One solution is to implement custom ContentHandler which would not write those new lines (still new lines from the original document will be kept):

public class OriginalBodyContentHandler extends BodyContentHandler {
    @Override
    public void ignorableWhitespace(char[] ch, int start, int length)
            throws SAXException {
        // Not writing extra new lines generated by XHTMLContentHandler.
    }
}
like image 89
Andrey Avatar answered Oct 20 '22 13:10

Andrey