Converting string to DateTime Polars

Question

I have a Polars dataframe with a column of type str with the date and time

df = pl.from_repr("""
┌─────────────────────────┐
│ EventTime               │
│ ---                     │
│ str                     │
╞═════════════════════════╡
│ 2020-03-02T13:10:42.550 │
└─────────────────────────┘
""")

I want to convert this column to the polars.Datetime type.

After reading this post Easily convert string column to pl.datetime in Polars, I came up with:

df = df.with_columns(pl.col('EventTime').str.to_datetime("%Y-%m-%dT%H:%M:%f", strict=False))

However, the values my column "EventTime' are all null.

Many Thanks!

Answer 1

You were close. You forgot the seconds component of your format specifier:

(
    df
    .with_columns(
        pl.col('EventTime')
        .str.to_datetime(
            format="%Y-%m-%dT%H:%M:%S%.f",
            strict=False)
        .alias('parsed EventTime')
    )
)

shape: (1, 2)
┌─────────────────────────┬─────────────────────────┐
│ EventTime               ┆ parsed EventTime        │
│ ---                     ┆ ---                     │
│ str                     ┆ datetime[ns]            │
╞═════════════════════════╪═════════════════════════╡
│ 2020-03-02T13:10:42.550 ┆ 2020-03-02 13:10:42.550 │
└─────────────────────────┴─────────────────────────┘

BTW, the format you are using is standard, so you can eliminate the format specifier altogether.

(
    df
    .with_columns(
        pl.col('EventTime')
        .str.to_datetime()
        .alias('parsed EventTime')
    )
)

shape: (1, 2)
┌─────────────────────────┬─────────────────────────┐
│ EventTime               ┆ parsed EventTime        │
│ ---                     ┆ ---                     │
│ str                     ┆ datetime[μs]            │
╞═════════════════════════╪═════════════════════════╡
│ 2020-03-02T13:10:42.550 ┆ 2020-03-02 13:10:42.550 │
└─────────────────────────┴─────────────────────────┘

Edit

And what if I would like to ignore the miliseconds? so the "%.f", if I just leave it out it can't interpret properly the dataframe

We need to allow Polars to parse the date string according to the actual format of the string.

That said, after the parsing, we can use dt.truncate to throw away the fractional part.

(
    df
    .with_columns(
        pl.col('EventTime')
        .str.to_datetime()
        .dt.truncate('1s')
        .alias('parsed EventTime')
    )
)

shape: (1, 2)
┌─────────────────────────┬─────────────────────┐
│ EventTime               ┆ parsed EventTime    │
│ ---                     ┆ ---                 │
│ str                     ┆ datetime[μs]        │
╞═════════════════════════╪═════════════════════╡
│ 2020-03-02T13:10:42.550 ┆ 2020-03-02 13:10:42 │
└─────────────────────────┴─────────────────────┘