Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Single quotes showing as diamond shaped question mark in browsers (no database or PHP)

Tags:

html

quotes

utf-8

I am working with a web page in which I switched the character set from iso-8859-1 to utf-8. The top of the page reads like this:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>[title of site]</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

I am only using ASCII characters in the page, and since utf-8 encoding supersets ASCII, this should be fine. However, single quotes in the text are showing up as question marks surrounded by black diamonds. I have verified these are are ASCII single quotes (not straight quotes).

I've read much online that describes solutions to the problem that involve PHP, magic quotes, database configuration, etc. However, this is a flat HTML page that isn't being rendered by any programs.

Also, many who have this problem are told to switch to UTF-8 to fix the problem. This is exactly how I introduced the problem.

Please look at http://mch.blackcatwebinc.com/src/events.html to see this problem.

like image 442
blackcatweb Avatar asked Aug 03 '12 03:08

blackcatweb


People also ask

Why am I seeing a black diamond with a question mark?

This is normally an encoding issue -- the symbol in question usually means "illegal character." Most commonly you see it when Latin-1 is being read as UTF-8, I think. In Safari, try switching View > Text Encoding to UTF-8 or Western (ISO Latin-1) or Western (MacRoman) and see if any of them makes it display correctly.

What is the meaning of using quotation marks in PHP?

Single or double quotes in PHP programming are used to define a string. But, there are lots of differences between these two. Single-quoted Strings: It is the easiest way to define a string. You can use it when you want the string to be exactly as it is written.

What are SQL quotes?

QUOTE() : This function in MySQL is used to return a result that can be used as a properly escaped data value in an SQL statement. The string is returned enclosed by single quotation marks and with each instance of backslash (\), single quote ('), ASCII NULL, and Control+Z preceded by a backslash.

What does a diamond shaped question mark mean in HTML?

The “diamond shaped question mark” is “�” U+FFFD REPLACEMENT CHARACTER and it indicates character-level data error: the browser has encountered bytes that do not represent a character at all in the character encoding being applied.

Are single quotes in the text showing up as question marks?

However, single quotes in the text are showing up as question marks surrounded by black diamonds. I have verified these are are ASCII single quotes (not straight quotes).

Is there a hex value for question mark diamonds in ASCII?

I saved the webpage and used PSPad hex view to view the page and found that where 17" and Let's words being displayed as question mark diamonds in the page weren't actually double quote (decimal 34) and single quote (decimal 39) in ASCII table, instead they are hex 94 and hex 92.

Why is there a question mark inside of a black diamond?

on some tables there is values that displayed as a black diamond with a question mark inside. I know this was a problem when importing data, that maybe the origing data table use different collation or encoding, so on my actual db there is this invalid character.


2 Answers

The only quotes in ASCII are the single quote ' (0x27 or 39) and the double quote " (0x22 or 33). What you have there is an 8-bit encoding that places quotes at 145 (0x91) and 146 (0x92) called CP1252; it's the standard 8-bit Western European encoding for Windows. If what you want is UTF-8, you need to convert that to UTF-8, since it's not valid UTF-8; valid UTF-8 uses multiple bytes for characters above 127 (0x7F), and places the opening and closing quotes at U+2018 and U+2019 respectively.

like image 143
prosfilaes Avatar answered Sep 28 '22 11:09

prosfilaes


According to the W3C, the meta charset

should appear as close as possible to the top of the head element

From http://www.w3.org/International/questions/qa-html-encoding-declarations#metacontenttype

So, I might try to place the meta tag above the title.

Also, as mentioned in the first answer by @user1505373, UTF is always capitalized and there is no space after the = in any of the examples I saw.

like image 26
Jason Gennaro Avatar answered Sep 28 '22 12:09

Jason Gennaro