pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 8000 ms exceeded

Question

I am getting error while running the below code. This is i'm using for web scraping.

from requests_html import HTMLSession

url = 'https://www.beerwulf.com/en-gb/c/beers?segment=Beers&catalogCode=Beer_1'

s = HTMLSession()
r = s.get(url)

r.html.render(sleep=1) 
print(r.status_code)

And i'm getting the below error

Traceback (most recent call last):
File "c:/Python/Python385/web_scraping/task1.py", line 8, in <module>
  r.html.render(sleep=1) 
File "C:\Python\Python385\lib\site-packages\requests_html.py", line 598, in render
  content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, 
  script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, 
  timeout=timeout, keep_page=keep_page))  
File "C:\Python\Python385\lib\asyncio\base_events.py", line 616, in run_until_complete
  return future.result()
File "C:\Python\Python385\lib\site-packages\requests_html.py", line 512, in _async_render
  await page.goto(url, options={'timeout': int(timeout * 1000)})
File "C:\Python\Python385\lib\site-packages\pyppeteer\page.py", line 885, in goto
  raise error
pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 8000 ms exceeded.

Answer 1

The default timeout for r.html.render() is 8 seconds. You get this error if all the JavaScript codes are not loaded within 8 seconds. You can fix this by changing the default timeout, let's say 20 seconds. Use the following code:

r.html.render(timeout=20)