Here's an array of datetime values:
array = np.array(['2016-05-01T00:00:59.3+10:00', '2016-05-01T00:02:59.4+10:00', '2016-05-01T00:03:59.4+10:00', '2016-05-01T00:13:00.1+10:00', '2016-05-01T00:22:00.5+10:00', '2016-05-01T00:31:01.1+10:00'], dtype=object)
pd.to_datetime
is very good at inferring datetime formats.
array = pd.to_datetime(array) print(array) DatetimeIndex(['2016-04-30 14:00:59.300000', '2016-04-30 14:02:59.400000', '2016-04-30 14:03:59.400000', '2016-04-30 14:13:00.100000', '2016-04-30 14:22:00.500000', '2016-04-30 14:31:01.100000'], dtype='datetime64[ns]', freq=None)
How can I dynamically figure out what datetime format pd.to_datetime
inferred? Something like: %Y-%m-%dT...
(sorry, my datetime foo is really bad).
Use datetime. strftime(format) to convert a datetime object into a string as per the corresponding format . The format codes are standard directives for mentioning in which format you want to represent datetime. For example, the %d-%m-%Y %H:%M:%S codes convert date to dd-mm-yyyy hh:mm:ss format.
I don't think it's possible to do this in full generality in pandas.
As mentioned in other comments and answers, the internal function _guess_datetime_format
is close to being what you ask for, but it has strict criteria for what constitutes a guessable format and so it will only work for a restricted class of datetime strings.
These criteria are set out in the _guess_datetime_format
function on these lines and you can also see some examples of good and bad formats in the test_parsing script.
Some of the main points are:
This means that it will fail to guess the format for datetime strings in the question despite them being a valid ISO 8601 format:
>>> from pandas.core.tools.datetimes import _guess_datetime_format_for_array >>> array = np.array(['2016-05-01T00:00:59.3+10:00']) >>> _guess_datetime_format_for_array(array) # returns None
In this case, dropping the timezone and padding the microseconds to six digits is enough to make pandas to recognise the format:
>>> array = np.array(['2016-05-01T00:00:59.300000']) # six digits, no tz >>> _guess_datetime_format_for_array(array) '%Y-%m-%dT%H:%M:%S.%f'
This is probably as good as it gets.
If pd.to_datetime
is not asked to infer the format of the array, or given a format string to try, it will just try and parse each string separately and hope that it is successful. Crucially, it does not need to infer a format in advance to do this.
First, pandas parses the string assuming it is (approximately) a ISO 8601 format. This begins in a call to _string_to_dts
and ultimately hits the low-level parse_iso_8601_datetime
function that does the hard work.
You can check if your string is able to be parsed in this way using the _test_parse_iso8601
function. For example:
from pandas._libs.tslib import _test_parse_iso8601 def is_iso8601(string): try: _test_parse_iso8601(string) return True except ValueError: return False
The dates in the array you give are recognised as this format:
>>> is_iso8601('2016-05-01T00:00:59.3+10:00') True
But this doesn't deliver what the question asks for and I don't see any realistic way to recover the exact format that is recognised by the parse_iso_8601_datetime
function.
If parsing the string as a ISO 8601 format fails, pandas falls back to using the parse()
function from the third-party dateutil library (called by parse_datetime_string
). This allows a fantastic level of parsing flexibility but, again, I don't know of any good way to extract the recognised datetime format from this function.
If both of these two parsers fail, pandas either raises an error, ignores the string or defaults to NaT
(depending on what the user specifies). No further attempt is made to parse the string or guess the format of the string.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With