PHP How to Remove/Replace Unknown question mark diamond characters

Question

I am using PHP to access data on old machines and output them.

Putty shows:

▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒NONE.
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

Its the weird formatting in a attempt to show data in a more clean way

PHP echo-ed chrome shows:

������
�NONE. �
������

I have tried:

$Str1 = str_replace("▒","",$Str1);

But it doesn't filter them out. The output is already utf 8.

Does anyone know how to filter out these things? Maybe identify what � is to php?

Answer 1

Try this:

$Str1 = preg_replace('/[\x00-\x1F\x7F-\xFF]/', '', $Str1);

Answer 2

The problem with a regex like '/[\x00-\x1F\x7F-\xFF]/' is that it simply demolishes all UTF-8. So, about only 1% or less of all possible characters will work with this. Full Working Demo

Why not prevent what is causing this?

With a fully utf-8-configured DB and proper headers, this problem can happen if you use:

• substr() — Instead, use mb_substr($utfstring, 0, 10, 'utf-8');.
• htmlspecialchars() — Instead, use htmlspecialchars($utfstring, ENT_QUOTES, 'UTF-8');.
• preg_replace() — Instead, use mb_ereg_replace().