Download .RData and .csv files from FTP using RCurl (or any other method)

Question

I've uploaded a .RData file (created using save()) to an ftp server, and I'm trying to use getURL() to download that same file. For all the examples and posts I've read, I can't seem to get this to work.

The .RData file was saved using:

save(results, file=RDataFilePath, compress="xz") #save object "results" w/ compression
#RDataFilePath is the location of the results.RData file on my harddrive

These data were uploaded using:

uploadURL <-"ftp://name:password@host/folder/results.RData" #name the url
ftpUpload(RDataFilePath, to=uploadURL, connecttimeout=120) #upload

This is how I try to download results.RData using getURL:

downloadURL <- "host/folder/results.RData"
load(getURL(downloadURL, userpwd="name:password", connecttimeout=120))

which gives the following error:

Error in curlPerform(curl = curl, .opts = opts, .encoding = .encoding) : 
  embedded nul in string: 'ý7zXZ'

When I paste the downloadURL string into my browser, the .RData file downloads immediately, so I know there isn't a typo. The error message suggests that the url can't get read b/c of the compression formatting; however, I get a similar error message when I use save() w/o compression.

I also get an error message when trying to download a .csv from the FTP:

read.csv(getURL(downloadURL1)) #downloadURL1 is similar to downloadURL, but points to the .csv file
Error in file(file, "rt") : cannot open the connection 

and then a warning which states In addition: Warning message: In file(file, "rt") : cannot open file and then starts listing the contents of the .csv.

I've been trying to figure this out for the better part of the morning, and I feel like I must be missing something really basic. I'm guessing that I need to change some curl option so that it knows what type of file it is going to read. My syntax is probably a bit off, and I'm not using getURL correctly, but I'm not sure what I should be doing.

Any tips would be greatly appreciated.

p.s. My current approach is based on this Post

Answer 1

You can try breaking it into two steps: first download the file, then load it.

download.file(downloadURL, "temp.rData")
load("temp.rData")

or sticking with rCurl you can try:

bin = getBinaryURL(downloadURL, ...yourOtherParams...) 
writeBin(bin, "temp.rData")  
load("temp.rData")

Answer 2

I spent quite a bit of time on this as well - hoping to use this in a Shiny app so I don't want to write to disk.

library(RCurl)
url <- "ftp://[email protected]/ALLF1Data.Rda"
userpwd <- "name:password"
bin = getBinaryURL(url, userpwd = userpwd, verbose = TRUE,
                   ftp.use.epsv = TRUE)

load(rawConnection(bin))

By Using rawConnection() I was able to avoid the write to disk step as it handled the RAW data type perfectly and avoided the error. FYI - This is my first post so I hope it's helpful