Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Setting Series as index

I'm using python 2.7 to take a numerical column of my dataframe data and make it an individual object (series) with an index of dates that is another column from data.

new_series = pd.Series(data['numerical_column'] , index=data['dates'])

However, when I do this, I get a bunch of NaN values in the Series:

dates
1980-01-31   NaN
1980-02-29   NaN
1980-03-31   NaN
1980-04-30   NaN
1980-05-31   NaN
1980-06-30   NaN
...

Why does my numerical_data values just disappear?

I realize that I can apparently achieve this goal by doing the following, although I'm curious why my initial approach failed.

new_series = data.set_index('dates')['numerical_column']
like image 617
zthomas.nc Avatar asked Oct 13 '16 19:10

zthomas.nc


1 Answers

I think there is problem with not align index of column data['numerical_column'].

So need convert it to numpy array by values:

new_series = pd.Series(data['numerical_column'].values , index=data['dates'])

Sample:

import pandas as pd
import datetime

data = pd.DataFrame({
'dates': {0: datetime.date(1980, 1, 31), 1: datetime.date(1980, 2, 29), 
          2: datetime.date(1980, 3, 31), 3: datetime.date(1980, 4, 30), 
          4: datetime.date(1980, 5, 31), 5: datetime.date(1980, 6, 30)}, 
'numerical_column': {0: 1, 1: 4, 2: 5, 3: 3, 4: 1, 5: 0}})
print (data)
        dates  numerical_column
0  1980-01-31                 1
1  1980-02-29                 4
2  1980-03-31                 5
3  1980-04-30                 3
4  1980-05-31                 1
5  1980-06-30                 0

new_series = pd.Series(data['numerical_column'].values , index=data['dates'])
print (new_series)
dates
1980-01-31    1
1980-02-29    4
1980-03-31    5
1980-04-30    3
1980-05-31    1
1980-06-30    0
dtype: int64

But method with set_index is nicer, but slowier:

#[60000 rows x 2 columns]
data = pd.concat([data]*10000).reset_index(drop=True)

In [65]: %timeit pd.Series(data['numerical_column'].values , index=data['dates'])
1000 loops, best of 3: 308 µs per loop

In [66]: %timeit data.set_index('dates')['numerical_column']
1000 loops, best of 3: 1.28 ms per loop

Verification:

If index of column has same index, it works nice:

s = data.set_index('dates')['numerical_column']
df = s.to_frame()
print (df)
            numerical_column
dates                       
1980-01-31                 1
1980-02-29                 4
1980-03-31                 5
1980-04-30                 3
1980-05-31                 1
1980-06-30                 0

new_series = pd.Series(df['numerical_column'] , index=data['dates'])
print (new_series)
dates
1980-01-31    1
1980-02-29    4
1980-03-31    5
1980-04-30    3
1980-05-31    1
1980-06-30    0
Name: numerical_column, dtype: int64
like image 118
jezrael Avatar answered Sep 28 '22 01:09

jezrael