Pandas Python - convert HH:MM:SS into seconds in aggegate (csv file)

Question

I'm trying to convert the numbers in 'Avg. Session Duration'(HH:MM:SS) column into whole numbers (in seconds) in Pandas read_csv module/function. For instance, '0:03:26' would be 206 seconds after the conversion.

Input example:

Source       Month  Sessions    Bounce Rate     Avg. Session Duration   
ABC.com     201501   408        26.47%           0:03:26 
EFG.com     201412   398        31.45%           0:04:03

I wrote a function:

def time_convert(x):
    times = x.split(':')
    return (60*int(times[0])+60*int(times[1]))+int(times[2])

This function works just fine while simply passing '0:03:26' to the function. But when I was trying to create a new column 'Duration' by applying the function to another column in Pandas,

df = pd.read_csv('myfile.csv')
df['Duration'] = df['Avg. Session Duration'].apply(time_convert)

It returned an Error Message:

> --------------------------------------------------------------------------- AttributeError                            Traceback (most recent call
> last) <ipython-input-53-01e79de1cb39> in <module>()
> ----> 1 df['Avg. Session Duration'] = df['Avg. Session Duration'].apply(lambda x: x.split(':'))
> 
> /Users/yumiyang/anaconda/lib/python2.7/site-packages/pandas/core/series.pyc
> in apply(self, func, convert_dtype, args, **kwds)    1991            
> values = lib.map_infer(values, lib.Timestamp)    1992 
> -> 1993         mapped = lib.map_infer(values, f, convert=convert_dtype)    1994         if len(mapped) and
> isinstance(mapped[0], Series):    1995             from
> pandas.core.frame import DataFrame
> 
> /Users/yumiyang/anaconda/lib/python2.7/site-packages/pandas/lib.so in
> pandas.lib.map_infer (pandas/lib.c:52281)()
> 
> <ipython-input-53-01e79de1cb39> in <lambda>(x)
> ----> 1 df['Avg. Session Duration'] = df['Avg. Session Duration'].apply(lambda x: x.split(':'))
> 
> AttributeError: 'float' object has no attribute 'split'

I don't know why it says values of 'Avg. Session Duration' are float.

Data columns (total 7 columns):
Source                   250 non-null object
Time                     251 non-null object
Sessions                 188 non-null object
Users                    188 non-null object
Bounce Rate              188 non-null object
Avg. Session Duration    188 non-null object
% New Sessions           188 non-null object
dtypes: object(7)

Can someone help me figure out where the problem is?

Answer 1

df['Avg. Session Duration'] should be strings for your function to work.

df =pd.DataFrame({'time':['0:03:26']})

def time_convert(x):
    h,m,s = map(int,x.split(':'))
    return (h*60+m)*60+s

df.time.apply(time_convert)

This works fine for me.

Answer 2

The error means that the column is recognized as float, not string. Fix the way you read the data e.g.:

#!/usr/bin/env python
import sys
import pandas

def hh_mm_ss2seconds(hh_mm_ss):
    return reduce(lambda acc, x: acc*60 + x, map(int, hh_mm_ss.split(':')))

df = pandas.read_csv('input.csv', sep=r'\s{2,}',
                     converters={'Avg. Session Duration': hh_mm_ss2seconds})
print(df)

Output

    Source   Month  Sessions Bounce Rate  Avg. Session Duration
0  ABC.com  201501       408      26.47%                    206
1  EFG.com  201412       398      31.45%                    243

[2 rows x 5 columns]

Answer 3

You can convert time to seconds with time and datetime from standard python library:

import time, datetime
def convertTime(t):
    x = time.strptime(t,'%H:%M:%S')
    return str(int(datetime.timedelta(hours=x.tm_hour,minutes=x.tm_min,seconds=x.tm_sec).total_seconds()))

convertTime('0:03:26') # Output 206.0
convertTime('0:04:03') # Output 243.0