How can I extract text visible on a page from its html source?

Question

I tried HtmlAgilityPack and the following code, but it does not capture text from html lists:

HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlStr);
HtmlNode node = doc.DocumentNode;
return node.InnerText;

Here is the code that fails:

<as html>
<p>This line is picked up <b>correctly</b>.  List items hasn't...</p>
<p><ul>
<li>List Item 1</li>
<li>List Item 2</li>
<li>List Item 3</li> 
<li>List Item 4</li>
</ul></p>
</as html>

Answer 1

Because you need walk over tree and concat in some way InnerText of all nodes

Answer 2

Following piece of code works for me:

string StripHTML(string htmlStr)
{
    HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
    doc.LoadHtml(htmlStr);
    var root = doc.DocumentNode;
    string s = "";
    foreach (var node in root.DescendantNodesAndSelf())
    {
        if (!node.HasChildNodes)
        {
            string text = node.InnerText;
            if (!string.IsNullOrEmpty(text))
            s += text.Trim() + " ";                     
        }
    }
    return s.Trim();
}