Reading information from a password protected site

Question

I have been using readLines() to scrape information from a website in an R tutorial. I now wish to extract data from my own website (specifically the awstats data) however the domain is password protected.

Is there a way that I can pass the url for the specific awstats data I require with a username and password.

the format of the url is:

http://domain.name:port/awstats.pl?month=02&year=2011&config=domain.name&lang=en&framename=mainright&output=alldomains

Thanks.

Answer 1

Formatting the url as http://username:password@domain... for use with download.file didn't work for me, but R.utils provides the function downloadFile that works perfectly:

require(R.utils)
downloadFile(myurl, myfile, username = "myusername", password ="mypassword")

See @joris-meys answer for a way to avoid including your username and password in plain text in your script.

EDIT Except it looks like downloadFile just reformats the URL to http://username:password@domain...? Hmm...