Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

utf8 characters wrongly displayed as question marks after conversion to mysqli_query()

My database is latin1_swedish_ci but all the tables which contain foreign characters (german, turkish...) are utf8_general_ci.

Before the upgrade to php 5.6, I used

mysql_query("SET CHARACTER SET utf8;");
mysql_query("SET NAMES utf8"); 

before mysql_query() and everything was displayed correctly in my page (<meta http-equiv="content-type" content="text/html;charset=UTF-8" /> in page header).

After the conversion of all mysql_query(...) to mysqli_query(id,...) and running under php 5.6, all the foreign languages are now scrambled with ? and �. Switching back to php 5.4 does not help. phpMyAdmin displays the mysql database (which has not changed) correctly.

I have looked around for a solution but nothing works... am I missing something?

What do I need to change in my code to work properly?

like image 610
Paul Godard Avatar asked Aug 04 '15 10:08

Paul Godard


2 Answers

After searching again and again...

MAMP MySQL not recognizing my.cnf values in OSX

Does MySQL included with MAMP not include a config file?

http://www.toptal.com/php/a-utf-8-primer-for-php-and-mysql

Here is the solution.

In my php scripts, I had to add a charset query after connecting to database.

$con=mysqli_connect($host",$user,$password,$db);
mysqli_set_charset($con,"utf8");

The previous charset & names I used with mysql_query() up to php 5.4 were not enough anymore.

mysqli_query($con,"SET CHARACTER SET utf8;");
mysqli_query($con,"SET NAMES utf8");

On my local server, I also had to recompile mysql after adding a my.cnf file containing the following lines :

[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
default-storage-engine = InnoDB
character-set-client-handshake
collation-server=utf8_general_ci
character-set-server=utf8
[mysqld_safe]
default-character-set=utf8

I also had to add the utf8 charset to MAMP by editing the MAMP/Library/share/charsets/index.xml and adding the folling lines :

<charset name="utf8">
  <family>Unicode</family>
  <description>UTF8 Unicode</description>
  <alias>utf8</alias>
  <collation name="utf8_general_ci" id="33">
   <flag>primary</flag>
   <flag>compiled</flag>
  </collation>
  <collation name="utf8_bin"        id="83">
    <flag>binary</flag>
    <flag>compiled</flag>
  </collation>
</charset>

On my web server, the 2 steps above were not necessary.

like image 83
Paul Godard Avatar answered Sep 20 '22 12:09

Paul Godard


These gibberish characters are the result of a browser or other user-visible software package rendering utf-8 characters as if they were ASCII or Latin-1.

EDIT In the Chrome browser, you can view the encoding with which your browser is rendering your page. Click the chrome menu (three little horizontal lines) in the upper right corner. Click More Tools> Click Encoding> Then you will see a choice of character sets. Try choosing a different one.

Try putting this line into your HTML documents' <HEAD> sections.

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

That may force the browser to use the correct character set.

END edit

After your switch to mysqli, did you keep the SET CHARACTER SET and SET NAMES queries as you opened up your mysql session? If not, put them back and see if it helps.

It's possible your database is working correctly but php is telling browsers you're using Latin-1. In fact that's very likely because phpmyadmin is working properly.

Try doing the things suggested here: Setting PHP default encoding to utf-8?

There's a lot of conceptual confusion around character sets and collations. When you say

My database is latin1_swedish_ci

you mean that the default character set for a newly defined table is latin1 and the default collation is case-insensitive Swedish. These are only defaults. What counts for data storage is the character set used for each column. You say that's utf8. The collation (utf8_general_ci) only counts for searching and matching.

like image 22
O. Jones Avatar answered Sep 20 '22 12:09

O. Jones