How to convert EST/EDT to GMT?

Question

I have a few records inside a column which represent either EST or EDT Time. I need to convert these times to GMT time. The format of the time are:

10/1/2010   0:0:0
10/1/2010   0:6:0
...
10/1/2010   23:54:0
...
10/3/2010   0:0:0
...

Can someone help me out here? thanks

Answer 1

The easiest, most reliable way I know to convert between timezones is to use the third-party pytz module:

import pytz
import datetime as dt

utc=pytz.utc
eastern=pytz.timezone('US/Eastern')
fmt='%Y-%m-%d %H:%M:%S %Z%z'

text='''\
10/1/2010   0:0:0
10/1/2010   0:6:0
10/1/2010   23:54:0
10/3/2010   0:0:0
'''

for datestring in text.splitlines():
    date=dt.datetime.strptime(datestring,"%m/%d/%Y %H:%M:%S")
    date_eastern=eastern.localize(date,is_dst=None)
    date_utc=date_eastern.astimezone(utc)
    print(date_utc.strftime(fmt))

yields:

2010-10-01 04:00:00 UTC+0000
2010-10-01 04:06:00 UTC+0000
2010-10-02 03:54:00 UTC+0000
2010-10-03 04:00:00 UTC+0000

Note however, your data does not specify if the datetime is in the EST or EDT timezone. There are some times which are ambiguous when you don't specify EST or EDT. For example, '10/27/2002 1:30:00' would be ambiguous:

>>> eastern.localize(datetime(2002, 10, 27, 1, 30, 00), is_dst=None)
AmbiguousTimeError: 2002-10-27 01:30:00

since this time happened twice due to Daylight Savings Time. Also some datetimes, like 2002-04-07 02:30:00, are nonexistent. See this link for a discussion of these and even more bizarre issues that arise when dealing with localtimes.

If you are willing to overlook these knotty corner cases, and if your machine is setup in the local timezone (e.g. EST/EDT), there is a way to convert between the local and UTC timezones which does not require the installation of pytz. The idea is to convert the datetime --> timetuple --> timestamp --> UTC datetime. The chain of conversions is done with

dt.datetime.utcfromtimestamp(time.mktime(date.timetuple()))

For example:

import time
import datetime as dt
import pytz

utc=pytz.utc
eastern=pytz.timezone('US/Eastern')
fmt='%Y-%m-%d %H:%M:%S %Z%z'

text='''\
10/1/2010   0:0:0
10/1/2010   0:6:0
10/1/2010   23:54:0
10/3/2010   0:0:0
3/13/2011   1:55:0
3/13/2011   3:00:0
'''
for datestring in text.splitlines():
    date=dt.datetime.strptime(datestring,"%m/%d/%Y %H:%M:%S")
    date_est=eastern.localize(date,is_dst=None)
    date_utc=date_est.astimezone(utc)
    date_utc2=dt.datetime.utcfromtimestamp(time.mktime(date.timetuple()))
    print('{d} --> {d_utc}    {d_utc2}'.format(
        d=date.strftime(fmt),
        d_utc=date_utc.strftime(fmt),
        d_utc2=date_utc2.strftime(fmt),
        ))
    assert date_utc.hour == date_utc2.hour

yields

2010-10-01 00:00:00 EDT-0400 --> 2010-10-01 04:00:00 UTC+0000    2010-10-01 04:00:00 
2010-10-01 00:06:00 EDT-0400 --> 2010-10-01 04:06:00 UTC+0000    2010-10-01 04:06:00 
2010-10-01 23:54:00 EDT-0400 --> 2010-10-02 03:54:00 UTC+0000    2010-10-02 03:54:00 
2010-10-03 00:00:00 EDT-0400 --> 2010-10-03 04:00:00 UTC+0000    2010-10-03 04:00:00 
2011-03-13 01:55:00 EST-0500 --> 2011-03-13 06:55:00 UTC+0000    2011-03-13 06:55:00 
2011-03-13 03:00:00 EDT-0400 --> 2011-03-13 07:00:00 UTC+0000    2011-03-13 07:00:00 

The last two dates tested above show the conversion works correctly even with times close to the switch between EST and EDT.

---

In summary, using the alternate method (without pytz), here is how to convert datetime objects representing local time to datetime objects representing GMT time, and vice versa:

In [83]: import datetime as dt
In [84]: import time
In [85]: import calendar

In [86]: date=dt.datetime(2010,12,1,0,0,0)    
In [87]: date
Out[87]: datetime.datetime(2010, 12, 1, 0, 0)

In [88]: date_utc=dt.datetime.utcfromtimestamp(time.mktime(date.timetuple()))    
In [89]: date_utc
Out[89]: datetime.datetime(2010, 12, 1, 5, 0)

In [90]: date_local=dt.datetime.fromtimestamp(calendar.timegm(date_utc.timetuple()))    
In [91]: date_local
Out[91]: datetime.datetime(2010, 12, 1, 0, 0)

Answer 2

Pseudocode for each record:

make a timestamp string: field[0].strip() + " " + field[1].strip()

use datetime.datetime.strptime() to convert that into a datetime.datetime instance

add a timedelta e.g. timedelta(hours=-4) to your timestamp

use timestamp.strftime() to produce whatever string representation you want for the output.

For the case where the time field is empty: If that means 0:0:0, modify the above to suit. If it means "time unknown", you'll need to do something else ...

Answer 3

Assume that we have a datetime string as "2019-04-09T23:59:55ET" in US/Eastern time. Here is the function to convert string to UTC:

from datetime import datetime
import pytz

eastern = pytz.timezone('US/Eastern')

def convent_est_to_utc(datetime_str):
    dt = datetime.strptime(datetime_str, '%Y-%m-%dT%H:%M:%SET')
    return dt.replace(tzinfo=eastern).astimezone(pytz.utc)

# testing
convent_est_to_utc("2019-04-09T23:59:55ET")

# The result: 2019-04-10 04:55:55+00:00

Answer 4

Without an associated time, the time zone doesn't matter ... nor can the date be translated to a different time zone. Is there a related time in another column?

EDIT: Alright, now that there IS a time, I'll let the python guru's take over. ;]

Answer 5

You can use pandas.DataFrame.tz_convert() like this:

import pandas as pd
from datetime import datetime

df = pd.read_csv("your_data_file_path.csv", index_col=False, engine='python')
df['Date'] = pd.to_datetime(df['Date'])
df['Date'] = df['Date'].dt.tz_localize('US/Eastern').dt.tz_convert('UTC')
df['Date'] = df['Date'].apply(lambda x: datetime.replace(x, tzinfo=None))

What the last row does is removing the timezone info from the datetime object, so you can operate with the date and time only (don't worry, that doesn't change the timezone again, it just strips it from the timestamp string).