Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way to read and write in-memory files in R?

I am trying to use R to analyze large DNA sequence files (fastq files, several gigabytes each), but the standard R interface to these files (ShortRead) has to read the entire file at once. This doesn't fit in memory, so it causes an error. Is there any way that I can read a few (thousand) lines at a time, stuff them into an in-memory file, and then use ShortRead to read from that in-memory file?

I'm looking for something like Perl's IO::Scalar, for R.

like image 583
Ryan C. Thompson Avatar asked Nov 08 '10 16:11

Ryan C. Thompson


2 Answers

I don’t know much about R, but have you had a look at the mmap package?

like image 64
Konrad Rudolph Avatar answered Sep 28 '22 04:09

Konrad Rudolph


It looks like ShortRead is soon to add a "FastqStreamer" class that does what I want.

like image 30
Ryan C. Thompson Avatar answered Sep 28 '22 04:09

Ryan C. Thompson