Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Optimus headless browser with C#

Tags:

c#

optimus

Can somebody tell me how to use Optimus (headless browser) nuget package with C# to get response from a URL. I also want javascript on the page to be executed automatically like phantomjs.

like image 579
Puneet Pant Avatar asked May 17 '17 11:05

Puneet Pant


People also ask

What are the best headless browsers for web developers?

Not all headless browsers are suited for the same testing scenarios, so you may have to try several different options to find the right combination of tools for your development needs. Take a look at these six popular choices for web developers. 1. Firefox headless mode

What are the main use cases for headless browsers?

The main use cases for headless browsers are: Taking screenshots of web pages. Running automated tests for JavaScript libraries. Automating interaction of web pages. Headless browsers are also useful for web scraping.

What is headless chrome?

Headless Chrome Example Headless Chrome provides support for Chrome versions 60 onwards and is available for Windows, Linux, and macOS. We need to download the.exe file of the latest version of the Chrome browser. Given below is the syntax for using Chrome in Headless mode:

Is PhantomJS still the most popular headless browser?

These browsers represent just a few of the testing environments available to developers. That being said, by taking the first 5 headless browsers mentioned in this post and comparing them with Google Trends, it's obvious that PhantomJS is still one of the most popular headless browser tools.


Video Answer


1 Answers

Quite a simple bit of kit:

  1. Create an Engine component first (common for dynamic and static pages):

    Engine engine = new Engine();

  2. Open the url of the html document you want to retreive:

    a) Not waiting for any elements added in with javascript:

    engine.OpenUrl("http://google.com").Wait();

    b) Waiting for any elements added in with javascript:

    engine.OpenUrl("http://google.com")

    and then either:

    • engine.WaitDesappearingOfId("some-id")
    • engine.WaitId("some-id")
    • engine.WaitDocumentLoad()
    • engine.WaitSelector("#some-id")
    • engine.WaitSelector(".some-class")

now you open the url, there are two ways of doing this - load the document (prior to any javascript being executed):

More complete examples:

public static string dynamicLoadingPage()
{
    var engine = new Engine();
    engine.OpenUrl("https://html5test.com");
    var tagWithValue = engine.WaitSelector("#score strong").FirstOrDefault();
    System.Console.WriteLine("Score: " + tagWithValue.InnerHTML);
}

Otherwise:

static string staticLoadingPage()
{
   var engine = new Engine();
   engine.OpenUrl("http://google.com").Wait();
   Console.WriteLine("The first document child node is: " + engine.Document.FirstChild);
   Console.WriteLine("The first document body child node is: " + engine.Document.Body.FirstChild);
   Console.WriteLine("The first element tag name is: " + engine.Document.ChildNodes.OfType<HtmlElement>().First().TagName);
   Console.WriteLine("Whole document innerHTML length is: " + engine.Document.DocumentElement.InnerHTML.Length);

}
like image 175
Sagar Patel Avatar answered Oct 30 '22 07:10

Sagar Patel