Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PowerShell - HTML parsing: get information from a website

Tags:

Update, Script is working with PowerShell V3.0, Thanks @ Doug

I want to use the following PowerShell script to get flight status information from Lufthansa. I can see flight status information in the browser, but I haven't found any way to access this information with my script.

I want to get the following information from the website:

  • flight status
  • time (departure, arrival)
  • airport (departure, arrival)
  • Flight Number (Only Lufthansa)

Script:

$flight = " LH3396" $url = "http://bing.com?q=flight status for $flight" $result = Invoke-WebRequest $url $elements = $result.AllElements | Where Class -eq "ans" | Select -First 1 -ExpandProperty innerText   #[string[]]$resultArray  $resultArray = @()  foreach($element in $elements.Split("`n")){     if($element.Length -gt "2")     {$resultArray += $element} } 
like image 800
LaPhi Avatar asked Jan 29 '12 13:01

LaPhi


People also ask

Can PowerShell scrape a website?

But, using a scripting language like PowerShell, a little ingenuity and some trial and error, it is possible to build a reliable web-scraping tool in PowerShell to pull down information from a lot of different web pages.

Can PowerShell read HTML?

PowerShell includes some great capabilities for working with two common forms of structured data: HTML and XML.

How do I browse a website using PowerShell?

There is no way of 'browsing' websites through Powershell, not in the traditional sense of the word anyway. However, you can download and upload content, create web service requests, login to websites, etc. through the web-client class and invoke-webrequest cmdlet.

How do I export output from PowerShell to HTML?

PowerShell provides a built-in cmdlet called ConvertTo-Html. This takes objects as input and converts each of them to an HTML web page. To use this, just take the output and pipe it directly to ConvertTo-Html. The cmdlet will then return a big string of HTML.


2 Answers

Here is a way to query Bing in PowerShell v3

function Get-FlightStatus {     param($query)      $url = "http://bing.com?q=flight status for $query"      $result = Invoke-WebRequest $url      $result.AllElements |          Where Class -eq "ans" |         Select -First 1 -ExpandProperty innerText     } 
Get-FlightStatus LH3102  Flight status for Lufthansa 3102  To depart · Jan 30, 2012  From: Hamburg (HAM) 05:35 PM terminal 2  To: Vienna (VIE) 07:05 PM   Bing Travel   Source: www.flightstats.com, 1 minute ago 
like image 171
Doug Finke Avatar answered Sep 19 '22 13:09

Doug Finke


You could use the Html Agility Pack.

Here's an article on using it with PowerShell: HTML Agility Pack Rocks Your Screen Scraping World

like image 38
TrueWill Avatar answered Sep 18 '22 13:09

TrueWill