Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get HTML Code from a website after it completed loading

I am trying to get the HTML Code from a specific website async with the following code:

var response = await httpClient.GetStringAsync("url");

But the problem is that the website usually takes another second to load the other parts of it. Which I need, so the question is if I can load the site first and read the content after a certain amount of time.

Sorry if this question already got answered, but I didn't really know what to search for.

Thanks, Twenty


Edit #1

If you want to try it yourself the URL is http://iloveradio.de/iloveradio/, I need the Title and the Artist which do not immediately load.

like image 558
Twenty Avatar asked Dec 22 '18 19:12

Twenty


People also ask

How do I extract the HTML code from a website?

View Source Using View Page Source Fire up Chrome and jump to the webpage you want to view the HTML source code. Right-click the page and click on “View Page Source,” or press Ctrl + U, to see the page's source in a new tab. A new tab opens along with all the HTML for the webpage, completely expanded and unformatted.

How do I view the HTML code of a website in Safari?

Open Safari. Navigate to the web page you would like to examine. Select the Develop menu in the top menu bar. Select the Show Page Source option to open a text window with the HTML source of the page.


2 Answers

You are on the wrong direction. The referenced site has playlist api which returns json. you can get information from :

http://iloveradio.de/typo3conf/ext/ep_channel/Scripts/playlist.php

Edit: Chome Inspector is used to find out Playlist link

enter image description here

like image 67
Derviş Kayımbaşıoğlu Avatar answered Sep 21 '22 13:09

Derviş Kayımbaşıoğlu


You could use Puppeteer-Sharp:

await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
using (var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = false }))
using (var page = await browser.NewPageAsync())
{
    await page.SetViewportAsync(new ViewPortOptions() { Width = 1280, Height = 600 });
    await page.GoToAsync("http://iloveradio.de/iloveradio/");
    await page.WaitForSelectorAsync("#artisttitle DIV");
    var artist = await page.EvaluateExpressionAsync<string>("$('#artisttitle DIV')[0].innerText");
    Console.WriteLine(artist);
    Console.ReadLine();
}
like image 36
hardkoded Avatar answered Sep 24 '22 13:09

hardkoded