Looping through node created by HtmlAgilityPack

Question

I need to parse this html code using HtmlAgilityPack and C#. I can get the div class="patent_bibdata" node, but I don'know how to loop thru the child nodes.

In this sample there are 6 hrefs, but I need to separate them into two groups; Inventors, Classification. I'm not interested in the last two. There can be any number of hrefs in this div.

As you can see there is a text before the two groups that says what the hrefs are.

code snippet

HtmlWeb hw = new HtmlWeb();
HtmlDocument doc = m_hw.Load("http://www.google.com/patents/US3748943");
string xpath = "/html/body/table[@id='viewport_table']/tr/td[@id='viewport_td']/div[@class='vertical_module_list_row'][1]/div[@id='overview']/div[@id='overview_v']/table[@id='summarytable']/tr/td/div[@class='patent_bibdata']";
HtmlNode node = m_doc.DocumentNode.SelectSingleNode(xpath);

So how would you do this?

<div class="patent_bibdata">
    <b>Inventors</b>:&nbsp;
    <a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Ronald T. Lashley
    </a>, 
    <a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Ronald T. Lashley
    </a><br>
    <b>Current U.S. Classification</b>:&nbsp;
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200P">84/312.00P</a>;
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200R">84/312.00R</a><br>
    <br>
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://patft.uspto.gov/netacgi/nph-Parser%3FSect2%3DPTO1%26Sect2%3DHITOFF%26p%3D1%26u%3D/netahtml/PTO/search-bool.html%26r%3D1%26f%3DG%26l%3D50%26d%3DPALL%26RefSrch%3Dyes%26Query%3DPN/3748943&usg=AFQjCNGKUic_9BaMHWdCZtCghtG5SYog-A">
    View patent at USPTO</a><br>
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://assignments.uspto.gov/assignments/q%3Fdb%3Dpat%26pat%3D3748943&usg=AFQjCNGbD7fvsJjOib3GgdU1gCXKiVjQsw">
    Search USPTO Assignment Database
    </a><br>
</div>

Wanted result InventorGroup =

<a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Ronald T. Lashley
    </a>
    <a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Thomas R. Lashley
    </a>

ClassificationGroup

<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200P">84/312.00P</a>;
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200R">84/312.00R</a>

The page I'm trying to scrape: http://www.google.com/patents/US3748943

// Anders

PS! I know that in this page the names of the inventors are the same, but in most of them they are different!

Answer 1

XPATH is your friend! Something like this will get you the inventors name:

HtmlWeb w = new HtmlWeb();
HtmlDocument doc = w.Load("http://www.google.com/patents/US3748943");
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//div[@class='patent_bibdata']/br[1]/preceding-sibling::a"))
{
    Console.WriteLine(node.InnerHtml);
}

Answer 2

So it's obvious that I don't understand XPath (yet). So I came up with this solution. Maybe not the smartest solution, but it works!

// Anders

List<string> inventorList = new List<string>();
List<string> classificationList = new List<string>();

string xpath = "/html/body/table[@id='viewport_table']/tr/td[@id='viewport_td']/div[@class='vertical_module_list_row'][1]/div[@id='overview']/div[@id='overview_v']/table[@id='summarytable']/tr/td/div[@class='patent_bibdata']";
HtmlNode nodes = m_doc.DocumentNode.SelectSingleNode(xpath);
bool bInventors = false;
bool bClassification = false;
for (int i = 0; i < nodes.ChildNodes.Count; i++)
{
    HtmlNode node = nodes.ChildNodes[i];
    string txt = node.InnerText;
    if (txt.IndexOf("Inventor") > -1)
    {
        bClassification = false;
        bInventors = true;
    }
    if (txt.IndexOf("Classification") > -1)
    {
        bClassification = true;
        bInventors = false;
    }
    if (txt.IndexOf("USPTO") > -1)
    {
        bClassification = false;
        bInventors = false;
    }
    string name = node.Name;
    if (name.IndexOf("a") > -1)
    {
        if (bInventors)
        {
            string inventor = node.InnerText;
            inventorList.Add(inventor);
        }
        if (bClassification)
        {
            string classification = node.InnerText;
            classificationList.Add(classification);
        }
    }