Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Feather format for long term storage since the release of apache arrow 1.0.1

As I'm given to understand due to the search of issues in the Feather Github, as well as questions in stackoverflow such as What are the differences between feather and parquet?, the Feather format was not recommended as long term storage due to Apache Arrow versions being 0.x.x, and considered volatile due to the continuous new releases.

My question is, has this situation changed as of the current Apache Arrow's version, 1.0.1? Is Feather considered stable to use as long term storage?

like image 596
Serelia Avatar asked Sep 27 '20 14:09

Serelia


People also ask

What is feather file format?

Feather is a fast, lightweight, and easy-to-use binary file format for storing data frames. It has a few specific design goals: Lightweight, minimal API: make pushing data frames in and out of memory as simple as possible. Language agnostic: Feather files are the same whether written by Python or R code.

Are feathers stable?

Feather files (using the v2 -- default -- format version, not the v1 "legacy" version) are stable starting with Apache Arrow 1.0.

What is Apache Pyarrow?

This is the documentation of the Python API of Apache Arrow. Apache Arrow is a development platform for in-memory analytics. It contains a set of technologies that enable big data systems to store, process and move data fast.


1 Answers

Feather files (using the v2 -- default -- format version, not the v1 "legacy" version) are stable starting with Apache Arrow 1.0.0.

like image 86
Wes McKinney Avatar answered Sep 28 '22 03:09

Wes McKinney