Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to read non-ASCII characters from CLI standard input

If I type å in CMD, fgets stop waiting for more input and the loop runs until I press ctrl-c. If I type a "normal" characters like a-z0-9!?() it works as expected.

I run the code in CMD under Windows 7 with UTF-8 as charset (chcp 65001), the file is saved as UTF-8 without bom. I use PHP 5.3.5 (cli).

<?php

echo "ÅÄÖåäö work here.\n";

while(1)
{
    echo '> '. fgets(STDIN);
}

?>

If I change charset to chcp 1252 the loop doesn't break when I type å and it print "> å" but the "ÅÄÖåäö work here" become "ÅÄÖåäö work here!". And I know that I can change the file to ANSI, but then I can't use special characters like ╠╦╗.

So why does fgets stop waiting for userinput after I have typed åäö?

And how can I fix this?

EDIT:

Also found a strange bug. echo "öäåÅÄÖåäö work here! Or?".chr(10); -> ��äåÅÄÖåäö work here! Or? re! Or?. If the first char in echo is å/ä/ö it print strange chars AND the end output duplicate's with n - 1 char.. (n = number of åäö in the begining of the string).

Eg: echo "åäö 1234" -> ??äö 123434 and echo åäöåäö 1234 -> ??äöåäö 1234 1234.

EDIT2 (solved):

The problem was chcp 65001, now I use chcp 437 (chcp 437). Big thanks to Timothy Martens!

like image 771
Sawny Avatar asked Nov 30 '11 18:11

Sawny


People also ask

How do I read non-ASCII characters in Python?

Approach 1: This approach is related to the inbuilt library unidecode. This library helps Transliterating non-ASCII characters in Python. It provides an unidecode() method that takes Unicode data and tries to represent it in ASCII.

How do I find non-ASCII characters in Excel?

There is no ActiveDocument object in Excel, which you can check by opening the VBE (ALT+F11) then opening the Object Browser (F2) and try a search.


1 Answers

Possible solution:

echo '>'; 
$line = stream_get_line(STDIN, 999999, PHP_EOL);

Notes: I was unable to reproduce your error using multiple versions of PHP. Using the following PHP version 5.3.8 gave me no issues

PHP 5.3 (5.3.8) VC9 x86 Non Thread Safe (2011-Aug-23 12:26:18) Arcitechture is Win XP SP3 32 bit

You might try upgrading PHP.

I downloaded php-5.3.5-nts-Win32-VC6-x86 and was not able to reproduce your error, it works fine for me.

Edit: Additionaly I typed the characters using my spanish keyboard.

Edit2:

CMD Command:

chcp 437

PHP Code:

<?php
$fp=fopen("php://stdin","r");
while(1){
    $str =  fgets(STDIN);
    echo mb_detect_encoding($str)."\n";
    echo '>'.stream_get_line($fp,999999,"\n")."\n";
}
?>

Output:

test
ASCII
test
>test
öïü

öïü
>öïü
like image 145
Timothy Martens Avatar answered Sep 30 '22 05:09

Timothy Martens