Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

unwanted type conversion in pandas.DataFrame.update

Tags:

python

pandas

Is there any reason why pandas changes the type of columns from int to float in update, and can I prevent it from doing it? Here is some example code of the problem

import pandas as pd
import numpy as np

df = pd.DataFrame({'int': [1, 2], 'float': [np.nan, np.nan]})

print('Integer column:')
print(df['int'])

for _, df_sub in df.groupby('int'):
    df_sub['float'] = float(df_sub['int'])
    df.update(df_sub)

print('NO integer column:')
print(df['int']) 
like image 909
Joerg Avatar asked Jul 01 '13 05:07

Joerg


People also ask

How do I change DataFrame data type in pandas?

In order to convert data types in pandas, there are three basic options: Use astype() to force an appropriate dtype. Create a custom function to convert the data. Use pandas functions such as to_numeric() or to_datetime()

How do I change the data type values in a column in pandas?

Change column type in pandas using DataFrame.apply() to_numeric, pandas. to_datetime, and pandas. to_timedelta as arguments to apply the apply() function to change the data type of one or more columns to numeric, DateTime, and time delta respectively.

How do you change a value type in a DataFrame?

The best way to convert one or more columns of a DataFrame to numeric values is to use pandas.to_numeric() . This function will try to change non-numeric objects (such as strings) into integers or floating-point numbers as appropriate.


1 Answers

here's the reason for this: since you are effectively masking certain values on a column and replace them (with your updates), some values could become `nan

in an integer array this is impossible, so numeric dtypes are apriori converted to float (for efficiency), as checking first is more expensive that doing this

a change of dtype back is possible...just not in the code right now, therefor this a bug (a bit non-trivial to fix though): github.com/pydata/pandas/issues/4094

like image 70
Jeff Avatar answered Nov 01 '22 18:11

Jeff