I am new to Python and I am facing problem in creating the Dataframe
in the format of key and value i.e.
data = [{'key':'\[GlobalProgramSizeInThousands\]','value':'1000'},]
Here is my code:
columnsss = ['key','value']; query = "select * from bparst_tags where tag_type = 1 "; result = database.cursor(db.cursors.DictCursor); result.execute(query); result_set = result.fetchall(); data = "["; for row in result_set: `row["tag_expression"]`) data += "{'value': %s , 'key': %s }," % ( `row["tag_expression"]`, `row["tag_name"]` ) data += "]" ; df = DataFrame(data , columns=columnsss);
But when I pass the data in DataFrame
it shows me
pandas.core.common.PandasError: DataFrame constructor not properly called!
while if I print the data and assign the same value to data variable then it works.
Convert PySpark Dataframe to Pandas DataFramePySpark DataFrame provides a method toPandas() to convert it to Python Pandas DataFrame. toPandas() results in the collection of all records in the PySpark DataFrame to the driver program and should be done only on a small subset of the data.
Method 1: Create Pandas DataFrame from a string using StringIO() One way to achieve this is by using the StringIO() function. It will act as a wrapper and it will help us to read the data using the pd. read_csv() function.
To create a dataframe, we need to import pandas. Dataframe can be created using dataframe() function. The dataframe() takes one or two parameters. The first one is the data which is to be filled in the dataframe table.
Just ran into the same error, but the above answer could not help me.
My code worked fine on my computer which was like this:
test_dict = {'x': '123', 'y': '456', 'z': '456'} df=pd.DataFrame(test_dict.items(),columns=['col1','col2'])
However, it did not work on another platform. It gave me the same error as mentioned in the original question. I tried below code by simply adding the list()
around the dictionary items, and it worked smoothly after:
df=pd.DataFrame(list(test_dict.items()),columns=['col1','col2'])
Hopefully, this answer can help whoever ran into a similar situation like me.
You are providing a string representation of a dict to the DataFrame constructor, and not a dict itself. So this is the reason you get that error.
So if you want to use your code, you could do:
df = DataFrame(eval(data))
But better would be to not create the string in the first place, but directly putting it in a dict. Something roughly like:
data = [] for row in result_set: data.append({'value': row["tag_expression"], 'key': row["tag_name"]})
But probably even this is not needed, as depending on what is exactly in your result_set
you could probably:
DataFrame(result_set)
read_sql_query
function to do this for you (see docs on this)If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With