Selenium Python: How to Scroll Down in a Pop Up window

Question

I am working on a Linkedin web scraping project. I am trying to get the list of companies that interest someone (notice I am not using the API). It is a dynamic website, so I would need to scroll down while scraping the names of the companies. I know how to do this in the MAIN window, but since Interest are a pop-up window this solution to scroll does not work. My code so far was:

from selenium.webdriver.common.keys import Keys
bar = driver.find_element_by_xpath('//ul[@class="entity-list row"]')
bar.send_keys(Keys.END)

Since it didn't work, I also tried:

bar = driver.find_element_by_xpath('//ul[@class="entity-list row"]')
driver.execute_script('arguments[0].scrollTop = arguments[0].scrollHeight', bar)

The problem is that I am not acting on the pop up window but in the main one so it has not the desired effect.

Answer 1

You can try to find element inside popup (the one that can be focused), for example some anchor:

element_inside_popup = driver.find_element_by_xpath('//div[@class="entity-list-wrapper ember-view"]//a')

and then use below code to scroll popup down:

from selenium.webdriver.common.keys import Keys

element_inside_popup.send_keys(Keys.END)

Answer 2

first I tried with this script but it doesn't work:

scrollable_popup = driver.find_element(By.XPATH, '/html/body/div[6]/div/div/div/div[2]')
for i in range(5):
    scrollable_popup.send_keys(Keys.END)
    time.sleep(2)

and then I used this script and it works fine with me:

scrollable_popup = driver.find_element(By.XPATH, '/html/body/div[6]/div/div/div/div[2]')
for i in range(5):
    driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", scrollable_popup)
    time.sleep(2)