Malformed UTF-8 character (fatal) at C:/Strawberry/perl/lib/Carp.pm line 324

Question

It's my source code:

use strict;
use warnings;
use diagnostics;
use utf8::all;

print join ' ',map{as_code_point($_)}sort qw/b 1 a A 日 本 国/;

sub as_code_point{
    my $char=shift;
    die"Only characters!"if length($char)>1;

    return "U+".uc sprintf "%04x",ord $char;
}

I don't know what's going wrong.I need your help.

Answer 1

That problem indicates the scalar contains an invalid value. That can happen when the source isn't encoded using UTF-8, but you tell Perl that it is (since it doesn't perform validity checks.)

If the file were properly encoded, you would not get that problem.

$ perl a.pl | perl
U+0031 U+0041 U+0061 U+0062 U+56FD U+65E5 U+672C

a.pl:

use strict;
use warnings;
use utf8::all;

my $chars = qq{qw/b 1 a A \x{65E5} \x{672C} \x{56FD}/};

while (<DATA>) {
    s/<<<CHARS>>>/$chars/g;
    print;
}

__DATA__
use strict;
use warnings;
use diagnostics;
use utf8::all;

print join ' ',map{as_code_point($_)}sort <<<CHARS>>>;
print "\n";

sub as_code_point{
    my $char=shift;
    die"Only characters!"if length($char)>1;

    return "U+".uc sprintf "%04x",ord $char;
}

Answer 2

My bet is that utf8::all is doing something that is breaking the loading of the Carp module. The first step is to see if there is indeed something wrong with your version of Carp. Try running this against the C:/Strawberry/perl/lib/Carp.pm:

perl -Mbytes -ne "printf qq/%02x [$_]\n/, ord for grep { ord > 127 } split //" C:/Strawberry/perl/lib/Carp.pm

You might have to escape the \ a second time (I don't have a windows machine to test against).

That should give you an idea of what non-ASCII bytes are in it. Check that those bytes are valid UTF-8 (it is possible you opened and saved the file with an editor that defaults to Window's native UTF-16).

Another option is to look at the line it is complaining about:

perl -ne 'print if $. == 324' C:/Strawberry/perl/lib/Carp.pm

Does anything look funny? You should also check the modified date on the file. Is it the same as the other core modules? If not, then it was likely edited.

If none of this finds the problem, get rid of use utf8::all; and slowly add back the functionality manually.