Broken pipe error selenium webdriver, when there is a gap between commands?

Question

Ubuntu 18.x + selenium webdriver(Firefox)

Facing a weird problem, the following block works if I run all of it together

from selenium import webdriver
url = 'https://indiamart.com'
driver = webdriver.Firefox()
driver.get(url)
driver.find_element_by_xpath(xpath).click()

This is happening with every url I have tried.

However if I execute one line at a time, it gives

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 326, in get
    self.execute(Command.GET, {'url': url})
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/remote_connection.py", line 472, in execute
    return self._request(command_info[0], url, body=data)
  File "/media/lib/python3.6/site-packages/selenium/webdriver/remote/remote_connection.py", line 495, in _request
    self._conn.request(method, parsed_url.path, body, headers)
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1065, in _send_output
    self.send(chunk)
  File "/usr/lib/python3.6/http/client.py", line 986, in send
    self.sock.sendall(data)
BrokenPipeError: [Errno 32] Broken pipe

This is the error on the code

driver.get(url)

However if I execute the same line again after the Broken Pipe error it works and gets the url.

I am very very confused. Can someone help me out.

Thanks

Answer 1

This is a known bug of the latest build v0.21.0 of geckodriver matched with the latest version of selenium v3.11

To work around this bug either: a) downgrade geckodriver to v0.20.1 or earlier b) wait for the bugfix/mitigations be rolled out in the upcoming versions of selenium and/or geckodriver

This bug originates from newly added support in v 0.21 of Keep-Alive feature. However, the default timeout from geckodriver in 0.21 is set to 5s, which is exceptionally short.

This bug is tracked here for geckodriver and here for selenium.

Answer 2

This error message...

BrokenPipeError: [Errno 32] Broken pipe

...implies that the GeckoDriver server process has received a SIGPIPE while writing to a socket. BrokenPipeError usually happens when a process tries to write to a socket which is fully closed on the client side. This may be happening when the GeckoDriver server process doesn't wait till all the data from the server is received and simply tries to close the socket (using close function)it had opened with the client.

Here you can find a detailed discussion on How to prevent errno 32 broken pipe?

Solution

• Moving forward as you are invoking click() on your desired element you need to induce WebDriverWait for the element to be clickable as follows:

  driver.get(url)
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "xpath"))).click()
  

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

• Again, the BrokenPipeError can also occur if your request is blocked or takes too long and after request-side (server) timeout. The server may close the connection and then, when the response-side (client) tries to write to the socket, it may throw a BrokenPipeError. Inthis cases you can set page_load_timeout as follows:

  driver.set_page_load_timeout(3)
  

Here you can find a detailed discussion on How to set the timeout of 'driver.get' for python selenium 3.8.0?

Answer 3

In the recent release, they have the issue, upgrade the selenium with latest release

pip install -U selenium