Converting string timestamp to datetime using pyarrow

Question

Is there any possibility to convert string timestamp in pyarrow table to datetime format before writing to parquet file?

Answer 1

Depending on the timestamp format, you can make use of pyarrow.compute.strptime function. It is not well-documented yet, but you can use something like this:

import pyarrow.compute as pc
pc.strptime(table.column("Timestamp"), format='%Y-%m-%d %H:%M:%S', unit='s')

provided your data is stored in table and "Timestamp" is the name of the column with timestamp strings.