Headless browser testing with download functionality?

Question

I've been looking for a solution to do headless testing in osx. But I need the ability to save files returned by the server.

I've tested selenium, phantomjs, casperjs and have looked into anything I could find online.

none of them supports downloading. am I missing something? are there any headless browser/testing frameworks that support downloads?

Answer 1

What you can do is:

• start a virtual display (see Xvfb)
• start up Firefox browser with preferences configured to automatically save csv files

Working example in python with additional comments (using pyvirtualdisplay xvfb wrapper):

from os import getcwd
import time

from pyvirtualdisplay import Display
from selenium import webdriver

# start the virtual display
display = Display(visible=0, size=(800, 600))
display.start()

# configure firefox profile to automatically save csv files in the current directory
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.download.dir", getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv")

browser = webdriver.Firefox(firefox_profile=fp)
browser.get('http://www.nationale-loterij.be/nl/onze-spelen/lotto/resultaten')

# check the option
browser.find_element_by_id('corporatebody_3_corporategrid93961a8f9b424ed6bd0697df356d9483_1_rblType_0').click()

# click the link
browser.find_element_by_name('corporatebody_3$corporategrid93961a8f9b424ed6bd0697df356d9483_1$btnDownload').click()

# hardcoded delay for waiting a file download (better check for the downloaded file to appear on the disk)
time.sleep(2)

# quit the browser
browser.quit()

# stop the display
display.stop()

See also:

• How do I run Selenium in Xvfb?
• Access to file download dialog in Firefox

Answer 2

I use on OSX selenium + wget command to perform downloads.

Here it is a sample of code:

new_driver = webdriver.Firefox()
new_driver.get(url)
for element in new_driver.find_elements_by_tag_name('img'):
    os.system('wget ' + element.get_attribute('src').rstrip('\n'))