For scrapy/selenium is there a way to go back to a previous page?

Question

I essentially have a start_url that has my javascript search form and button, hence the need of selenium. I use selenium to select the appropriate items in my select box objects, and click the search button. The following page, I do some scrapy magic. However, now I want to go BACK to the original start_url and fill out a different object, etc. and repeat until no more.

Essentially, I have tried making a for-loop and trying to get the browser to go back to the original response.url, but somehow it crashed. I may try having a duplicate list of start_url's on the top for scrapy to parse through, but I'm not sure if that is the best approach. What can I do in my situation?

Answer 1

Here the advice is to use driver.back() : https://selenium-python.readthedocs.io/navigating.html#navigation-history-and-location

Answer 2

The currently selected answer provides a link to an external site and that link is broken. The selenium docs talk about

driver.forward() driver.back() 

but those will sometimes fail, even if you explicitly use some wait functions.

I found a better solution. You can use the below command to navigate backwards.

driver.execute_script("window.history.go(-1)") 

hope this helps someone else in the future.