Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas DataFrame cast multiple types to columns

Tags:

python

pandas

I'd like to declare different types for the columns of a pandas DataFrame at instantiation:

frame = pandas.DataFrame({..some data..},dtype=[str,int,int])

This works if dtype is only one type (e.g dtype=float), but not multiple types as above - is there a way to do this?

The common solution seems to be to cast later:

frame['some column'] = frame['some column'].astype(float)

but this has a couple of issues:

  1. It's messy
  2. Looks like it involves an unnecessary copy operation - this could be expensive on large data sets.
like image 207
Mike Vella Avatar asked Jun 02 '14 16:06

Mike Vella


People also ask

Can a pandas column have multiple data types?

Pandas uses other names for data types than Python, for example: object for textual data. A column in a DataFrame can only have one data type.

How do you change multiple column types in pandas?

Change column type in pandas using DataFrame.apply() to_numeric, pandas. to_datetime, and pandas. to_timedelta as arguments to apply the apply() function to change the data type of one or more columns to numeric, DateTime, and time delta respectively.

Can we create a DataFrame with multiple data types in Python?

You can create a DataFrame from multiple Series objects by adding each series as a columns. By using concat() method you can merge multiple series together into DataFrame. This takes several params, for our scenario we use list that takes series to combine and axis=1 to specify merge series as columns instead of rows.

Can pandas series have different data types?

Series is a one-dimensional labeled array capable of holding data of any type (integer, string, float, python objects, etc.).


2 Answers

You can also create a NumPy array with specific dtypes and then convert it to DataFrame.

data = np.zeros((2,),dtype=[('A', 'i4'),('B', 'f4'),('C', 'a10')])
data[:] = [(1,2.,'Hello'),(2,3.,"World")]
DataFrame(data)

See From structured or record array

like image 110
drastega Avatar answered Nov 15 '22 17:11

drastega


As an alternative, you can specify the dtype for each column by creating the Series objects first.

In [2]: df = pd.DataFrame({'x': pd.Series(['1.0', '2.0', '3.0'], dtype=float), 'y': pd.Series(['1', '2', '3'], dtype=int)})

In [3]: df
Out[3]: 
   x  y
0  1  1
1  2  2
2  3  3

[3 rows x 2 columns]

In [4]: df.dtypes
Out[4]: 
x    float64
y      int64
dtype: object
like image 23
R. Max Avatar answered Nov 15 '22 17:11

R. Max