Write R data as csv directly to s3

Question

I would like to be able to write data directly to a bucket in AWS s3 from a data.frame\ data.table object as a csv file without writing it to disk first using the AWS CLI.

obj.to.write.s3 <- data.frame(cbind(x1=rnorm(1e6),x2=rnorm(1e6,5,10),x3=rnorm(1e6,20,1)))

at the moment I write to csv first then upload to an existing bucket then remove the file using:

fn <- 'new-file-name.csv'
write.csv(obj.to.write.s3,file=fn)
system(paste0('aws s3 ',fn,' s3://my-bucket-name/',fn))
system(paste0('rm ',fn))

I would like a function that writes directly to s3? is that possible?

Answer 1

In aws.s3 0.2.2 the s3write_using() (and s3read_using()) functions were added.

They make things much simpler:

s3write_using(iris, FUN = write.csv,
                    bucket = "bucketname",
                    object = "objectname")