Google Cloud Platform - Deploy a Cloud Function that starts a webdriver

Question

I am defining a Cloud Function on GCP for scraping a website in Python.

I am starting simple by defining a function that simply opens the webdriver:

from selenium import webdriver

def launch_search(request):
    # Starting a webdriver
    driver = webdriver.Chrome()
    return 'Success'

This function doesn't work (Error: could not handle the request when I trigger it), probably because the Chrome Driver is not installed on my remote machine. Therefore:

• How can I install it?
• Or can I scrape a webpage using Selenium, without opening the page with a webdriver?

Answer 1

Given that Cloud Functions are Serverless, you are unable to control the server machine. You can use other manageable services such as GCE or GKE

Answer 2

You can't currently use Python to run Selenium scripts. There's a Feature Request in the Public Issue Tracker currently open for this that can be found here.

As an alternative, you can use Node.JS with Puppeteer. I found this blog post that details a use-case.