I am trying to tab through an Instagram page given a user input. I am able to get to the page. The page loads, then the class is found, then the code breaks. Here is my code:
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import urllib.request, urllib.parse, urllib.error
import time
def get_posts(tag, user_count):
'''Getting html of page to be scrape for
hrefs that will get me the user names'''
print("getting page")
url = "https://www.instagram.com/explore/tags/" + tag + "/"
try:
driver = webdriver.Chrome()
driver.get(url)
print("successfully requested site")
except:
print("Unable to reach site")
quit()
browser = driver.find_element_by_class_name('_si7dy')
actions = ActionChains(browser)
for i in range(user_count):
actions = actions.send_keys(Keys.TAB)
time.sleep(0.5)
actions.perform()
soup = BeautifulSoup(driver.page_source, 'lxml')
try:
posts = soup.find_all("div", class_ = ["_mck9w","_gvoze","_f2mse"])
except:
print("No links found")
quit()
print("Length of posts: ",(len(posts)))
print(len(posts))
print(type(posts))
print("All Done")
driver.close()
return posts
I keep getting this error:
packages\selenium\webdriver\common\action_chains.py", line 69, in __init__
if self._driver.w3c:
AttributeError: 'WebElement' object has no attribute 'w3c'
I have searched around but have not found any info on w3c. I have never tabbed down a page before and so am using the answer found here: Send multiple tab key presses with selenium.
ActionChains seems like the best way to tab multiple times down the page, but if anyone has a better method I am open to trying that.
ActionChains should receive WebDriver, but you are sending WebElement instead
actions = ActionChains(driver)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With