Python => ValueError: unsupported format character 'Y' (0x59)

Question

I don't understand the ValueError with Y. I escape with %...

table = town+"_history"
db.execute("SELECT DATE_FORMAT(snapdate,'%%Y-%%m-%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM %s WHERE blockid =%%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid))

Answer 1

You escape the %% but then use the string as a formatter first:

"...." % table,

that returns a new string with the %% escaped percentages replaced by single % characters. The MySQL database adapter (ab)uses string formatting with % too, so it'll take that output and expect to be able to fill %s slots with escaped SQL literals. It is there that your '%Y-%m-%d' part of the SQL statement is being interpreted again as a string format and the error is thrown.

The solution is to either double the doubling:

db.execute("SELECT DATE_FORMAT(snapdate,'%%%%Y-%%%%m-%%%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM %s WHERE blockid = %%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid,))

or use str.format() instead and avoid having to doubly-escape:

db.execute("SELECT DATE_FORMAT(snapdate,'%%Y-%%m-%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM {0} WHERE blockid = %s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7".format(table), (blockid,))

Here {0} is replaced by the table name and the %% escapes are left untouched; the database adapter will use the %s slot to fill in the blockid parameter and return a SQL statement with the %% escapes turned into single % characters.

Answer 2

Finally, @Martijn Pieters you are completely right. And thank you for your useful answer. The other error comes from SUM and COUNT. Python sometimes runs in a crazy way when dealing with JSON. So the complete answer is:

db.execute("SELECT DATE_FORMAT(snapdate,'%%%%Y-%%%%m-%%%%d') AS date, CAST(SUM( population ) AS CHAR ) AS accountpopulation, CAST(count( blockid ) AS CHAR) AS number_block FROM %s WHERE blockid = %%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid,))