I am using python3 and pandas to connect to some sql database:
import pandas as pd
import mysql.connector
cnx = mysql.connector.connect(user='me', password='***',
host='***',
database='***')
df=pd.read_sql("select id as uid,refType from user where registrationTime>=1451606400",con=cnx)
cnx.close()
I am getting 2 columns: id and refType, both of them are of type string (varchar in SQL terminology). However, for some reason, refType column is correctly imported as string, but uid column is imported as bytearray. This is how they look:
df.head()
uid
0 [49, 54, 54, 57, 55, 54, 50, 55, 64, 97, 110]
1 [49, 54, 54, 57, 55, 54, 50, 56, 64, 105, 111]
2 [49, 48, 49, 53, 51, 50, 51, 50, 57, 53, 57, 5...
3 [57, 53, 52, 52, 56, 57, 56, 56, 49, 50, 57, 5...
4 [49, 54, 54, 57, 55, 54, 50, 57, 64, 105, 111]refType
0 adx_Facebook.IE_an_ph_u8_-.cc-ch.g-f.au-ret7.c...
1 adx_Facebook.IE_io_ph_u4_-.cc-gb.g-f.au-toppay...
2 ad_nan_1845589538__CAbroadEOScys_-.cc-ca.g-f.a...
3 ad_offerTrialPay-DKlvl10-1009
4 adx_Facebook.IE_io_ph_u4_-.cc-us.g-f.au-topspe...
And this is how uid column is supposed to look:
[i.decode() for i in df['uid'][1:5]]
['16697628@io', '10153232959751867@fb', '954489881295911@fb', '16697629@io']
I don't understand neither why was it converted to bytearray nor how to choose to convert it to string. I couldn't find anything about it or similar questions in internet or pandas documentation. Of course, I can always convert that column to string after importing, but that is not preferred, because the shown sql query is just an example, and in real table there can be hundreds of columns that would be incorrectly imported as bytearrays. It would be real pain in the ass to manually find those columns and convert to string
The connector itself outputs the same bytearray:
cursor = cnx.cursor()
cursor.execute('select id as uid,refType from user where registrationTime>=1451606400 LIMIT 1')
cursor.fetchall()`
[(bytearray(b'16697627@an'), 'adx_Facebook.IE_an_ph_u8_-.cc-ch.g-f.au-ret7.cr-cys.dt-all.csd-291215.-')
The data types of the columns in SQL database are "Varchar(32)" for the first column (uid) and "Varchar(128)" for the second one (refType)
The MySQLdb documentation states (scroll down a little further to threadsafety): The general upshot of this is: Don't share connections between threads. It's really not worth your effort or mine, and in the end, will probably hurt performance, since the MySQL server runs a separate thread for each connection.
The connect() constructor creates a connection to the MySQL server and returns a MySQLConnection object.
Python needs a MySQL driver to access the MySQL database. In this tutorial we will use the driver "MySQL Connector". We recommend that you use PIP to install "MySQL Connector". PIP is most likely already installed in your Python environment.
Had the same issue with the package "mysql-connector". Installing "mysql-connector-python" instead did the trick for me.
pip install mysql-connector-python
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With