Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the best format in which to save data frames to disc in R for storage?

What is the best format to persist simple data frames to disc in R for storage while limiting semantic loss?

I ask because I'm archiving a data set. In an ideal world, my data format would have the follow characteristics:

  1. Stability - the storage format will be compatible with future version of R
  2. Semantic compatibility - the storage format will understand the semantics of R's primative data types. For example, it will be able to store ordered factors with labels in a sensible manner.
  3. Open standard - ideally, the format will be an open standard so other statistics packages (now or in the future) will be able to understand it

My first thought was to use CSV which is very stable, but lacks the semantic richness required. On the other hand, R's builtin RData format completely captures R's semantics, but seems likely to change between releases (correct me if I'm wrong).

Is there another format that finds a balance between these three imperatives?

like image 525
fmark Avatar asked Mar 09 '13 06:03

fmark


1 Answers

Dump it to a text file with dput. That way you get all the structure of R's objects, and its in a text-based form that, should R stop existing, can be parsed fairly easily.

It probably doesn't pass (3), your 'open standard' test.

R is pretty good for backward compatibility with its .RData format, so even if the files written by the latest R aren't the same as older ones, the latest R will still read old files. However, if R should stop existing, reverse-engineering of the binary format is orders of magnitude harder than grokking the output from dput.

like image 68
Spacedman Avatar answered Oct 27 '22 13:10

Spacedman