Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to read in ISO 8859-1 (Latin-1) encoded text in Perl

So I'm trying to write a perl script to read in a file encoded in Latin-1. For some reason, this just isn't working out. When I try to do a simple search for a character that I know is in the file (it's in the first line), nothing shows up. I'm using use encoding "iso 8859-1"; below, but I've also tried binmode(STDIN, ":utf8");. Any suggestions on what I might be doing wrong, and how to make it right?

use encoding "iso 8859-1";

while(<>)
{
    if(/ó/gi)
    {
    print "Found one!\n";
    }
}
like image 424
John Montgomery Avatar asked Nov 19 '10 01:11

John Montgomery


People also ask

What is encoding =' Latin 1?

Latin-1, also called ISO-8859-1, is an 8-bit character set endorsed by the International Organization for Standardization (ISO) and represents the alphabets of Western European languages.

What is the difference between UTF-8 and ISO 8859-1?

UTF-8 is a multibyte encoding that can represent any Unicode character. ISO 8859-1 is a single-byte encoding that can represent the first 256 Unicode characters. Both encode ASCII exactly the same way.

How do I encode a string in Perl?

$octets = encode_utf8($string); Equivalent to $octets = encode("utf8", $string); The characters that comprise $string are encoded in Perl's internal format and the result is returned as a sequence of octets. All possible characters have a UTF-8 representation so this function cannot fail.


1 Answers

Don’t use the use encoding pragma: it’s broken.

Either specify the encoding here:

use open ":encoding(Latin1)";

or put it in the open itself:

open(FH, "< :encoding(Latin1)", $pathname)
   || die "can't open $pathname: $!";

or binmode it after opening:

binmode(FH, ":encoding(Latin1)")
   || die "can't binmode to encoding Latin1";

If you’re using <ARGV>, then use open is probably easiest.

Don’t forget to set the encoding on your output streams, too.

like image 181
tchrist Avatar answered Sep 30 '22 05:09

tchrist