Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Replace NaT in a dataframe with a previous variable

how can I replace NaT from a dataframe with a date/variable that was created (setup) before?

I got the following dataframe:

     Date            Name         CF
0        NaT         Peter       -10
0 2017-12-14         Peter       -20
1        NaT         Tomas       -5.5
1 2017-12-15         Peter       -25
2        NaT         Tomas       -3
2 2017-12-14         Tomas       -5
3        NaT         Walker      -4.6
3 2017-12-15         Tomas        88
4 2017-12-15         Tomas       -30
5 2017-12-15         Walker       15

I would like to replace NaT with a variable that was setup at the beginning of my Project: anEnd variable like this:

format_date = "%d-%m-%Y"
end = datetime.date(2017, 12, 15)

I tried two approaches: First, I used np.where from Pandas, but it does not recognize the NaT. Second, I tried this

test = table_final2.Date.astype(object).where(table_final2.Date.notnull(), end)

but this results in a mix of formats for the column Date

         Date              Name      CF
0           2017-12-15     Peter    -10
0  2017-12-14 00:00:00     Peter    -20
1           2017-12-15     Tomas    -5.5
1  2017-12-15 00:00:00     Peter    -25
2           2017-12-15     Tomas    -3
2  2017-12-14 00:00:00     Tomas    -5
3           2017-12-15     Walker   -4.6
3  2017-12-15 00:00:00     Tomas     88
4  2017-12-15 00:00:00     Tomas    -30
5  2017-12-15 00:00:00     Walker    15

So, how can I replace NaT accurately? Thanks in advance!

PD: when test.dtypes shows Date as an object.

like image 458
Newbie Avatar asked Dec 26 '17 23:12

Newbie


People also ask

How do you replace a value in a DataFrame with another value?

The replace() method replaces the specified value with another specified value. The replace() method searches the entire DataFrame and replaces every case of the specified value.

How do you replace a specific value in a Pandas DataFrame?

DataFrame. replace() function is used to replace values in column (one value with another value on all columns). This method takes to_replace, value, inplace, limit, regex and method as parameters and returns a new DataFrame. When inplace=True is used, it replaces on existing DataFrame object and returns None value.

Is NaT same as NaN?

NaN doesn't equal NaN . And NaT doesn't equal NaT . But None does equal None . Now let's turn our attention finding missing values.


1 Answers

IIUC you can use Series.fillna() method:

In [138]: end = pd.to_datetime('2017-12-15')

In [139]: df['Date'] = df['Date'].fillna(end)

In [140]: df
Out[140]:
        Date    Name    CF
0 2017-12-15   Peter -10.0
0 2017-12-14   Peter -20.0
1 2017-12-15   Tomas  -5.5
1 2017-12-15   Peter -25.0
2 2017-12-15   Tomas  -3.0
2 2017-12-14   Tomas  -5.0
3 2017-12-15  Walker  -4.6
3 2017-12-15   Tomas  88.0
4 2017-12-15   Tomas -30.0
5 2017-12-15  Walker  15.0
like image 115
MaxU - stop WAR against UA Avatar answered Oct 07 '22 00:10

MaxU - stop WAR against UA