Remotely Scrape Page and Get most Relevant title or Description for Images with XPath

Question

What I'm looking at doing is essentially the same thing a Tweet button or Facebook Share / Like button does, and that is to scrape a page and the most relevant title for a piece of data. The best example I can think of is when you're on the front page of a website with many articles and you click a Facebook Like Button. It will then get the proper information for the post relative to (nearest) the Like button. Some sites have Open Graph tags, but some do not and it still works.

Since this is done remotely, I only have control of the data that I want to target. In this case the data are images. Rather than retrieving just the <title> of the page, I am looking to somehow traverse the dom in reverse from the starting point of each image, and find the nearest "title". The problem is that not all titles occur before an image. However, the chance of the image occurring after the title in this case seems fairly high. With that said, it is my hope to make it work well for nearly any site.

Thoughts:

• Find the "container" of the image and then use the first block of text.
• Find the blocks of text in elements that contain certain classes ("description", "title") or elements (h1,h2,h3,h4).

Title backups:

• Using Open Graph Tags
• Using just the <title>
• Using ALT tags only
• Using META Tags

Summary: Extracting the images isn't the problem, it's how to get relevant titles for them.

Question: How would you go about getting relevant titles for each of the images? Perhaps using DomDocument or XPath?

Answer 1

Your approach seems good enough, I would just give certain tags / attributes a weight and loop through them with XPath queries until I find something that exits and it's not void. Something like:

i = 0

while (//img[i][@src])
  if (//img[i][@alt])
    return alt
  else if (//img[i][@description])
    return description
  else if (//img[i]/../p[0])
    return p
  else
    return (//title)

  i++

---

A simple XPath example (function ported from my framework):

function ph_DOM($html, $xpath = null)
{
    if (is_object($html) === true)
    {
        if (isset($xpath) === true)
        {
            $html = $html->xpath($xpath);
        }

        return $html;
    }

    else if (is_string($html) === true)
    {
        $dom = new DOMDocument();

        if (libxml_use_internal_errors(true) === true)
        {
            libxml_clear_errors();
        }

        if ($dom->loadHTML(ph()->Text->Unicode->mb_html_entities($html)) === true)
        {
            return ph_DOM(simplexml_import_dom($dom), $xpath);
        }
    }

    return false;
}

And the actual usage:

$html = file_get_contents('http://en.wikipedia.org/wiki/Photography');

print_r(ph_DOM($html, '//img')); // gets all images
print_r(ph_DOM($html, '//img[@src]')); // gets all images that have a src
print_r(ph_DOM($html, '//img[@src]/..')); // gets all images that have a src and their parent element
print_r(ph_DOM($html, '//img[@src]/../..')); // and so on...
print_r(ph_DOM($html, '//title')); // get the title of the page