Locating table with no id or class attributes

Question

I am trying to scrape a site with a few tables. Neither table has a class or an id and the site really doesn't use either one so I am not sure if there is a way for me to get the data. Here is the link to the site - I would post the html but it would be too long.

http://epi.hbsna.com/products/dept.asp?msi=0&sid=6076533CE8C648AE9883BDDBED795B29&dept_id=315&parent_id=0

The table I am trying to extract begins on line 310.

Answer 1

Since this is BeautifulSoup specific question, here is a working BeautifulSoup specific solution. The idea is to find the element having the SKU# text and locate the first table parent:

import requests
from bs4 import BeautifulSoup


data = requests.get('http://epi.hbsna.com/products/dept.asp?msi=0&sid=6076533CE8C648AE9883BDDBED795B29&dept_id=315&parent_id=0').content
soup = BeautifulSoup(data, "html.parser")

table = soup.find(text="SKU#").find_parent("table")
for row in table.find_all("tr")[1:]:
    print([cell.get_text(strip=True) for cell in row.find_all("td")])

Prints the contents of the table:

['40010001', 'ABA Service Kit', '-', '1-1/4" 10', 'None', '5-1/2"', '0.63', 'Clamp', '42710566']
['40010002', 'ABA Service Kit', '-', '1-1/4" 10', '5/8" RH', '5-1/2"', '0.63', 'Clamp', '42710566']
...
['40010649', 'ABA Service Kit', '-', '1 1/2 - 10', '1.5', '6"', '0.50', 'Strap', '427-10517']
['40050604', 'ABA Service Kit', 'none', '1 1/2" - 10"', '1 1/2" LH', '6"', '0.50', 'Strap', '427-10601']

Answer 2

How do you feel about using this xpath expression?

//*[./text()="SKU#"]/ancestor::table[1]

It means, "find the first element with text being exactly SKU#, then select its closest table ancestor."

You can try it out in a browser inspector by passing the expression as a string to the $x function.

---

See this answer for working with xpath in beautifulsoup.