Filling missing values using forward and backward fill in pandas dataframe (ffill and bfill)

Question

Beginner with panda dataframes. I have this data set below with missing values for column A and B (Test.csv):

DateTime              A             B 01-01-2017 03:27         01-01-2017 03:28         01-01-2017 03:29    0.18127718  -0.178835737 01-01-2017 03:30    0.186923018 -0.183260853 01-01-2017 03:31         01-01-2017 03:32         01-01-2017 03:33    0.18127718  -0.178835737 

I can use this code to fill in values using forward propagation, but this only fills in for 03:31 and 03:32, and not 03:27 and 03:28.

import pandas as pd import numpy as np  df = pd.read_csv('test.csv', index_col = 0) data = df.fillna(method='ffill') ndata = data.to_csv('test1.csv') 

results in:

   DateTime              A             B     01-01-2017 03:27             01-01-2017 03:28             01-01-2017 03:29    0.18127718  -0.178835737     01-01-2017 03:30    0.186923018 -0.183260853     01-01-2017 03:31    0.186923018 -0.183260853     01-01-2017 03:32    0.186923018 -0.183260853     01-01-2017 03:33    0.18127718  -0.178835737 

How could I include the 'Bfill' to fill in the missing values for 03:27 and 03:28 using the backfil?

Answer 1

You can use ffill and bfill if need replace NaN values forward and backward filling:

print (df)                          A         B DateTime                             01-01-2017 03:27       NaN       NaN 01-01-2017 03:28       NaN       NaN 01-01-2017 03:29  0.181277 -0.178836 01-01-2017 03:30  0.186923 -0.183261 01-01-2017 03:31       NaN       NaN 01-01-2017 03:32       NaN       NaN 01-01-2017 03:33  0.181277 -0.178836  data = df.ffill().bfill() print (data)                          A         B DateTime                             01-01-2017 03:27  0.181277 -0.178836 01-01-2017 03:28  0.181277 -0.178836 01-01-2017 03:29  0.181277 -0.178836 01-01-2017 03:30  0.186923 -0.183261 01-01-2017 03:31  0.186923 -0.183261 01-01-2017 03:32  0.186923 -0.183261 01-01-2017 03:33  0.181277 -0.178836 

Which is same as the function fillna with parameters:

data = df.fillna(method='ffill').fillna(method='bfill')