Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using lambda and strftime on dates when there are null values (Pandas)

I'm trying to change the format of a datetime column in my Dataframe using lambda and strftime like below

df['Date Column'] = df['Date Column'].map(lambda x: x.strftime('%m/%d/%Y'))

However, since I have null values in some of these fields, this is giving me an error. I cannot drop these null rows because I still need them for the data in the other columns. Is there a way around this error without dropping the nulls.

Perhaps something like

df['Date Column'].map(lambda x: x.strftime('%m/%d/%Y') if x != null else "")

?

The method I've used is to drop the nulls, format the column, then merge it back onto the original dataset, but this seems like a very inefficient method.

like image 303
FortuneFaded Avatar asked Feb 18 '16 17:02

FortuneFaded


2 Answers

You should be not checking for nan/nat (un)equality, but .notnull() should work and it does for me:

s = pd.date_range('2000-01-01', periods=5).to_series().reset_index(drop=True)
s[2] = None
s

0   2000-01-01
1   2000-01-02
2          NaT
3   2000-01-04
4   2000-01-05
dtype: datetime64[ns]

s.map(lambda x: x.strftime('%m/%d/%Y') if pd.notnull(x) else '')

0    01/01/2000
1    01/02/2000
2              
3    01/04/2000
4    01/05/2000
dtype: object

This returns the same that the answers by @Alexander and @Batman but is more explicit. It may also be slightly slower for large series.

Alternatively you can use the .dt accesor. The null values will be formatted as NaT.

s.dt.strftime('%m/%d/%Y')

0    01/01/2000
1    01/02/2000
2           NaT
3    01/04/2000
4    01/05/2000
dtype: object
like image 158
Stop harming Monica Avatar answered Sep 18 '22 16:09

Stop harming Monica


Personally I'd just define a small function, and then use that.

def to_string(date):
    if date:
        string = date.strftime('%Y%m%d')
    else:
        string = ""

    return string

Then

df['Date Column'].map(to_string) 

Otherwise

df['Date Column'].map(lambda x: x.strftime('%Y%m%d') if x else "")
like image 35
Batman Avatar answered Sep 16 '22 16:09

Batman