Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

strip HTML Tags with perl

Whats the easiest way to strip the HTML tags in perl. I am using a regular expression to parse HTML from a URL which works great but how can I strip the HTML tags off?

Here is how I am pulling my HTML

 #!/usr/bin/perl -w
use strict;
use warnings;
use LWP::Simple;
my $now_string = localtime;

my $html = get("http://www.spc.noaa.gov/climo/reports/last3hours.html")
    or die "Could not fetch NWS page.";
$html =~ s/<script.*?<\'/script>/sg;
$html =~ s/<.+?>//sg;
$html =~ m{(Hail Reports.*)Wind Reports}s || die;
my @hail = $1;
like image 800
shinjuo Avatar asked Jul 05 '10 01:07

shinjuo


People also ask

How do I strip a tag in HTML?

How do you remove your HTML Code from a given HTML URL? Users can copy and paste HTML code using the view source of the URL, or click on the URL button and enter the URL and click on Strip HTML Button.

How do I strip a string in HTML?

To strip out all the HTML tags from a string there are lots of procedures in JavaScript. In order to strip out tags we can use replace() function and can also use . textContent property, . innerText property from HTML DOM.

What does it mean to strip HTML?

stripHtml( html ) Changes the provided HTML string into a plain text string by converting <br> , <p> , and <div> to line breaks, stripping all other tags, and converting escaped characters into their display values.


1 Answers

An attempt to answer your misguided question


Problems


It's a bad habit to get into regex'ing out HTML because there are so many rules and ways to get around them, that may eventually open your code up to hacking techniques. While you might have a legitimate need for something simple now, it is very easy to reuse code and forget why it was a bad idea to reuse it, especially when you don't add comments like # This code is NOT secure and should not be used to parse HTML anywhere else!!! or # Christina Alguilera writes songs based on this code!!!

Example of differences in HTML that require lots of regex rules:

<div>...</div>
<div style="blah">
<div style="background:url(../div)">
<div style=".." class='noticesinglequote'>

The list goes on and that's only for well-formed HTML. Some other examples of problems include:

  1. HTML elements closed improperly (eg <div><span></div></span>) or not at all
  2. Spelling errors (eg <dvi>..</div>)
  3. HTML designed with the intention to break your script
  4. Other issues: comments, whitespaces, charsets, etc

Solution


You may have accepted an answer, but you should look at XML::Parser and HTML::TreeBuilder.

Rather than stripping out parts of the HTML Document, you are probably more interested in drilling down to the part of the document you want (eg everything in <body> or a certain div inside of it), which is why you most likely want something that one of the above modules provide. Not to mention, parsers can be used to do their best at removing all HTML elements and returning only text/CData.

like image 104
vol7ron Avatar answered Sep 28 '22 19:09

vol7ron