Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

RSelenium on docker: where are files downloaded?

I am using Selenium using a docker image:

require(RSelenium)

  if (length(system("docker ps -l", intern = TRUE))<2)
    try({system("docker run -d -p 4445:4444 selenium/standalone-firefox:2.53.0")})

It works, I can connect to any url and navigate. However when I click a button to download a file, it sometimes saves it (partially, saved as xxxxxxx.csv.part) to /tmp/mozilla_mozillaUser0, and sometimes to ... nowhere, or maybe another location I cannot find...

Is there a reason for that?

Also I tried to open the driver using

makeFirefoxProfile(list(browser.download.dir = "D:/temp"))

but it returns a weird error

Error in file(tmpfile, "rb") : cannot open the connection
In addition: Warning messages:
1: running command '"zip" -r9Xjq "C:\Users\rocks\AppData\Local\Temp\RtmpoPhjUb\file31076202d4f.zip" "C:\Users\rocks\AppData\Local\Temp\RtmpoPhjUb/firefoxprofile/prefs.js" ' had status 127 
2: In file(tmpfile, "rb") :
  cannot open file 'C:\Users\rocks\AppData\Local\Temp\RtmpoPhjUb\file31076202d4f.zip': No such file or directory

where I can understand why this desn't work given that all links are in windows but my selenium runs in a docker container or Ubuntu.

My Setup: R running on Windows, and I have a docker image of Ubuntu that contains the selenium server.

EDIT: turns out my issue was because firefox was not installed on the host machine (which is not needed in theory). I fixed the issue by using (instead of invoking makeFirefoxProfile):

fprof = structure(list(firefox_profile = "UEsDBBQAAAAIANJiVEobimJN8QAAABkCAAAIAAAAcHJlZnMuanOFkT9PwzAQxXckvkOUCSQnlliZkDqygcSITHyJTW3fyXdNIj49btqBoLTd7t57v/OfOzDkT8rQP9RfGafStRanFNDY1vpcq6rWDiNohnDM6t3Z5frx+f7uGt5jsJBfPUuZ8nQrHU0yQxHY4fThIL2JyeLTUNjeBIYLvINAkF+IuE0wlor3LZsR3nHneX+8fjwE8VSG6bn58aQMUfCdEY9J/+tPiabDWA5hBrs2LxmRJ8xrqeNRCcyyFD6Wl2lKQ3UuvwkGVf3Nk+1VtQBOYjihFIxPqlrlYO4grNExlc/jZsOZt8XIGzJ2AtKwZDBx2ewvUEsBAj8AFAAAAAgA0mJUShuKYk3xAAAAGQIAAAgAJAAAAAAAAAAgAAAAAAAAAHByZWZzLmpzCgAgAAAAAAABABgAzaOo9TCL0gHdkgMtLYvSAd2SAy0ti9IBUEsFBgAAAAABAAEAWgAAABcBAAAAAA=="), .Names = "firefox_profile")
remDr <- remoteDriver(extraCapabilities = ePrefs, port = 4445)
like image 538
RockScience Avatar asked Dec 11 '22 12:12

RockScience


1 Answers

The docker container is a separate entity to the HOST which is running it. You need to map a directory on the HOST to a directory on the container you download files to:

You can do this when starting your container:

docker run -d -p 4445:4444 -p 5901:5900 -v /home/john/test:/home/seluser/Downloads selenium/standalone-firefox-debug:2.53.1

Here (i am running docker on linux) I have mapped a directory on my linux HOST (/home/john/test) to a directory on the container (/home/seluser/Downloads).

We then need to add the necessary information to the firefox profile:

library(RSelenium)
ePrefs <- makeFirefoxProfile(
  list(
    browser.download.dir = "/home/seluser/Downloads",
    "browser.download.folderList" = 2L,
    "browser.download.manager.showWhenStarting" = FALSE,
    "browser.helperApps.neverAsk.saveToDisk" = "multipart/x-zip,application/zip,application/x-zip-compressed,application/x-compressed,application/msword,application/csv,text/csv,image/png ,image/jpeg, application/pdf, text/html,text/plain,  application/excel, application/vnd.ms-excel, application/x-excel, application/x-msexcel, application/octet-stream"))
remDr <- remoteDriver(extraCapabilities = ePrefs, port = 4445)
remDr$open()
remDr$navigate("http://www.colorado.edu/conflict/peace/download/")
firstzip <- remDr$findElement("xpath", "//a[contains(@href, 'zip')]")
firstzip$clickElement()

We can check if the download is on the HOST machine:

> list.files("/home/john/test/")
[1] "peace.zip"
like image 120
jdharrison Avatar answered Dec 22 '22 02:12

jdharrison