How do I convert Pandas DataFrame to a Huggingface Dataset object?

Question

I have the following df:

import pandas as pd
df = pd.DataFrame({"foo": ["bar", "baz"]})

How do I convert to a Huggingface Dataset?

Answer 1

datasets have an easy way to convert pandas dataframes to hugginface datasets:

from datasets import Dataset
dataset = Dataset.from_pandas(df)

# if you want to go back to a pandas dataframe
df = dataset.to_pandas()

more info here: https://huggingface.co/docs/datasets/main/en/loading#inmemory-data