Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas.DataFrame.shift() fill_value not working

This may sound like a rather weird question, but when I'm using the pandas.DataFrame.shift() function with the fill_value keyword argument, I keep getting a TypeError: shift() got an unexpected keyword argument 'fill_value'.

Even a simple example on the documentation is giving me that error:

df = pd.DataFrame({'c1': [1, 2, 3], 
                   'c2': [4, 5, 6],
                   'c3': [7, 8, 9]})

df.shift(periods=1, fill_value=0)

Would normally expected

    c1    c2    c3
0   0     0     0
1   1     4     7
2   2     5     8
3   3     6     9

but it throws the error. Would anybody happen to know what the problem may be? I've searched elsewhere but it seems that nobody else is experiencing this problem...

like image 770
Sean Avatar asked Apr 13 '19 06:04

Sean


People also ask

How do I move rows down in pandas DataFrame?

Shift an Entire Dataframe Using Pandas Shift One of the Pandas . shift() arguments is the periods= argument, which allows us to pass in an integer. The integer determines how many periods to shift the data by. If the integer passed into the periods= argument is positive, the data will be shifted down.

How does shift work in pandas?

shift() function Shift index by desired number of periods with an optional time freq. This function takes a scalar parameter called the period, which represents the number of shifts to be made over the desired axis. This function is very helpful when dealing with time-series data.

How do you shift rows in a data frame?

shift() If you want to shift your column or subtract the column value with the previous row value from the DataFrame, you can do it by using the shift() function. It consists of a scalar parameter called period, which is responsible for showing the number of shifts to be made over the desired axis.


1 Answers

Problem is use pandas bellow 0.24+ where is not imlemented this parameter in DataFrame.shift.

fill_value : object, optional

The scalar value to use for newly introduced missing values. the default depends on the dtype of self. For numeric data, np.nan is used. For datetime, timedelta, or period data, etc. NaT is used. For extension dtypes, self.dtype.na_value is used.

Changed in version 0.24.0.

Then use DataFrame.fillna:

df = df.shift(periods=1).fillna(0)

Or DataFrame.fillna with specify first row by position by DataFrame.iloc if possible another missing values in data and necessary replace only it:

df = df.shift(periods=1)
df.iloc[0] = df.iloc[0].fillna(0)
print (df)
    c1   c2   c3
0  0.0  0.0  0.0
1  1.0  4.0  7.0
2  2.0  5.0  8.0
like image 63
jezrael Avatar answered Oct 01 '22 21:10

jezrael