Python, Beautiful Soup + how to parse dynamic class?

Question

I am new to Beautiful Soup and Python in general, but my question is how would I go about specifying a class that is dynamic (productId)? Can I use a mask or search part of the class, i.e. "product summary*"

<li class="product_summary clearfix {productId: 247559}">

</li>

I want to get the product_info and also the product_image (src) data below the product_summary class list, but I don't know how to find_all when my class is dynamic. Hope this makes sense. My goal is to insert this data into a MySQL table, so my thought is I need to store all data into variables at the highest (product summary) level. Thanks in advance for any help.

from bs4 import BeautifulSoup
from urllib.request import Request, urlopen

url = Request('http://www.shopwell.com/sodas/c/22', headers={'User-Agent': 'Mozilla/5.0'})
webpage = urlopen(url).read()

soup = BeautifulSoup(webpage)

product_info = soup.find_all("div", {"class": "product_info"})

for item in product_info:

        detail_link = item.find("a", {"class": "detail_link"}).text

        try:
            detail_link_h2 = ""
            detail_link_h2 = item.h2.text.replace("\n", "")
        except:
            pass

        try:
            detail_link_h3 = ""
            detail_link_h3 = item.h3.text.replace("\n", "")
        except:
            pass
        try:
            detail_link_h4 = item.h4.text.replace("\n", "")
        except:
            pass

        print(detail_link_h2 + ", " + detail_link_h3 + ", " + detail_link_h4)


product_image = soup.find_all("div", {"class": "product_image"})

for item in product_image:

        img1 = item.find("img")
        print(img1)

Answer 1

I think you can use regular expressions like this:

import re
product_image = soup.find_all("div", {"class": re.compile("^product_image")})