Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas shift converts my column from integer to float.

shift converts my column from integer to float. It turns out that np.nan is float only. Is there any ways to keep shifted column as integer?

df = pd.DataFrame({"a":range(5)})
df['b'] = df['a'].shift(1)

df['a']
# 0    0
# 1    1
# 2    2
# 3    3
# 4    4
# Name: a, dtype: int64

df['b']

# 0   NaN
# 1     0
# 2     1
# 3     2
# 4     3
# Name: b, dtype: float64
like image 796
user3226167 Avatar asked Jan 26 '17 09:01

user3226167


People also ask

How do you change a column from float to int?

To convert a column that includes a mixture of float and NaN values to int, first replace NaN values with zero on pandas DataFrame and then use astype() to convert. Use DataFrame. fillna() to replace the NaN values with integer value zero. Yields below output.

How do pandas turn into floats?

pandas Convert String to FloatUse pandas DataFrame. astype() function to convert column from string/int to float, you can apply this on a specific column or on an entire DataFrame. To cast the data type to 54-bit signed float, you can use numpy. float64 , numpy.

What does shift do in pandas?

shift() function Shift index by desired number of periods with an optional time freq. This function takes a scalar parameter called the period, which represents the number of shifts to be made over the desired axis. This function is very helpful when dealing with time-series data.


1 Answers

Solution for pandas under 0.24:

Problem is you get NaN value what is float, so int is converted to float - see na type promotions.

One possible solution is convert NaN values to some value like 0 and then is possible convert to int:

df = pd.DataFrame({"a":range(5)})
df['b'] = df['a'].shift(1).fillna(0).astype(int)
print (df)
   a  b
0  0  0
1  1  0
2  2  1
3  3  2
4  4  3

Solution for pandas 0.24+ - check Series.shift:

fill_value object, optional
The scalar value to use for newly introduced missing values. the default depends on the dtype of self. For numeric data, np.nan is used. For datetime, timedelta, or period data, etc. NaT is used. For extension dtypes, self.dtype.na_value is used.

Changed in version 0.24.0.

df['b'] = df['a'].shift(fill_value=0)
like image 146
jezrael Avatar answered Oct 17 '22 02:10

jezrael