Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to make a proxy authentication in headless Chrome

I'm trying to use a headless Chrome browser through a proxy that requires authentication:

require "selenium-webdriver"

options = Selenium::WebDriver::Chrome::Options.new(
      args: ["headless", "proxy-server=http://#{host}:#{port}"]
    )
driver = Selenium::WebDriver.for(:chrome, options: options)

This works when authentication is not needed, but now I need to use it with authentiation.

In "List of Chromium Command Line Switches" I didn't find how to pass username and password correctly. In Google I found this option, but it doesn’t work:

options = Selenium::WebDriver::Chrome::Options.new(
      args: ["headless", "proxy-server=http://#{username}:#{password}@#{host}:#{port}"]
    )

How do I use headless-chrome with a proxy authentication?

like image 947
Daniel Pogosyan Avatar asked Feb 25 '20 15:02

Daniel Pogosyan


People also ask

What is proxy authentication?

The HTTP Proxy-Authenticate response header defines the authentication method that should be used to gain access to a resource behind a proxy server. It authenticates the request to the proxy server, allowing it to transmit the request further.

Can headless Chrome be detected?

Nonetheless, instrumenting Headless Chrome with a framework such as Puppeteer will still leave traces that make it possible to detect it as a non-human user.


1 Answers

I have been searching for solution for almost 2 weeks. It was in 2019 and I haven't even figured it out. As far as I know you can't set proxy with username/password. It was possible before with extension, but now this way is not available. There is a way to do it with webdrivers gem:
If there is a proxy between you and the Internet then you will need to configure the gem to use the proxy. You can do this by calling the configure method.

Webdrivers.configure do |config|
  config.proxy_addr = 'myproxy_address.com'
  config.proxy_port = '8080'
  config.proxy_user = 'username'
  config.proxy_pass = 'password'
end

However I couldn't make it work in my project. The only thing that worked is setting whitelist with my IP on the Proxy server so I do not need to set username/password. So it works without credentials, only like this:

    Capybara.register_driver :headless_chrome do |app|
      client = Selenium::WebDriver::Remote::Http::Default.new
      client.read_timeout = 60
      capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
        'goog:chromeOptions' => { args: %w[no-sandbox headless disable-gpu disable-dev-shm-usage
                                           window-size=1280,1024 enable-features=NetworkService,NetworkServiceInProcess] },
        'loggingPrefs' => { browser: 'ALL', client: 'ALL', driver: 'ALL', server: 'ALL' }
      )
      capabilities['goog:chromeOptions'][:args] << "user-agent=#{user_agent}" if user_agent
      capabilities['goog:chromeOptions'][:args] << "proxy-server=http://#{proxy[:ip]}:#{proxy[:port]}" if proxy

      Capybara::Selenium::Driver.new(app, browser: :chrome, desired_capabilities: capabilities, http_client: client)
    end

I know my answer is not you have been looking, but maybe it will help you in some way or push you to correct answer. Good luck

like image 66
Roman Alekseiev Avatar answered Sep 30 '22 18:09

Roman Alekseiev