Problem

You want to remove HTML tags from a string, leaving just plain text.

Solution

The following oft-cited solution is simple but wrong on all but the most trivial HTML:

($plain_text = $html_text) =~ s/<[^>]*>//gs;     #WRONG

A correct but slower and slightly more complicated way is to use the CPAN modules:

use HTML::Parse; use HTML::FormatText; $plain_text = HTML::FormatText->new->format(parse_html($html_text));

Discussion

As with almost everything else, there is more than one way to do it. Each solution attempts to strike a balance between speed and flexibility. Occasionally you may find HTML that's simple enough that a trivial command line call will work:

% perl -pe 's/<[^>]*>//g' file

However, this will break on with files whose tags cross line boundaries, like this:

<IMG SRC = "foo.gif"      ALT = "Flurp!">

So, you'll see people doing this instead:

% perl -0777 -pe 's/<[^>]*>//gs' file

or its scripted equivalent:

{     local $/;               # temporary whole-file input mode     $html = <FILE>;     $html =~ s/<[^>]*>//gs; }

But even that isn't good enough except for simplistic HTML without any interesting bits in it. This approach fails for the following examples of valid HTML (among many others):

<IMG SRC = "foo.gif" ALT = "A > B">  <!-- <A comment> -->  <script>if (a<b && a>c)</script>  <# Just data #>  <![INCLUDE CDATA [ >>>>>>>>>>>> ]]>

If HTML comments include other tags, those solutions would also break on text like this:

<!-- This section commented out.     <B>You can't see me!</B> -->

The only solution that works well here is to use the HTML parsing routines from CPAN. The second code snippet shown above in the Solution demonstrates this better technique.

For more flexible parsing, subclass the HTML::Parser class and only record the text elements you see:

package MyParser; use HTML::Parser; use HTML::Entities qw(decode_entities);  @ISA = qw(HTML::Parser);  sub text {     my($self, $text) = @_;     print decode_entities($text); }  package main; MyParser->new->parse_file(*F);

If you're only interested in simple tags that don't contain others nested inside, you can often make do with an approach like the following, which extracts the title from a non-tricky HTML document:

($title) = ($html =~ m#<TITLE>\s*(.*?)\s*</TITLE>#is);

Again, the regex approach has its flaws, so a more complete solution using LWP to process the HTML is shown in Example 20.4 .

#!/usr/bin/perl # htitle - get html title from URL  die "usage: $0 url ...\n" unless @ARGV; require LWP;  foreach $url (@ARGV) {     $ua = LWP::UserAgent->new();     $res = $ua->request(HTTP::Request->new(GET => $url));     print "$url: " if @ARGV > 1;     if ($res->is_success) {         print $res->title, "\n";     } else {         print $res->status_line, "\n";     } }

Here's an example of the output:

% htitle http://www.ora.com www.oreilly.com -- Welcome to O'Reilly & Associates!  % htitle http://www.perl.com/ http://www.perl.com/nullvoid http://www.perl.com/: The www.perl.com Home Page http://www.perl.com/nullvoid: 404 File Not Found

See Also

The documentation for the CPAN modules HTML::TreeBuilder, HTML::Parser, HTML::Entities, and LWP::UserAgent; Recipe 20.5