Menu
Is it possible to parse data from web html page using windows batch?
let's say I have a web page: www.domain.com/data/page/1 Page source html:
...
<div><a href="/post/view/664654"> ....
....
In this case I would need get /post/view/664654 from web page.
My idea is to loop through www.domain.com/data/page/1 ... # (to some given number) and extract all the /post/view's. Then I would have a list of links, and from each of those links I would extract href values (either images or videos).
So far I was only successful in downloading image or video when I know exact link, using wget. But I don't know how (if possible at all) to parse html data.
edit
<body>
<nav>
<section>links I dont need</section>
</nav>
<article>
<section>links I need</section>
</article>
485
It's better to parse structured markup as a hierarchical object, rather than scraping as flat text. That way you aren't so dependent upon the formatting of the data you're parsing (whether it's minified, spacing has changed, whatever).
The batch language isn't terribly well-suited to parse markup language like HTML, XML, JSON, etc. In such cases, it can be extremely helpful to use a hybrid script and borrow from JScript or PowerShell methods to scrape the data you need. Here's an example demonstrating a batch + JScript hybrid script. Save it with a .bat extension and give it a run.
@if (@CodeSection == @Batch) @then
@echo off & setlocal
set "url=http://www.domain.com/data/page/1"
for /f "delims=" %%I in ('cscript /nologo /e:JScript "%~f0" "%url%"') do (
rem // do something useful with %%I
echo Link found: %%I
)
goto :EOF
@end // end batch / begin JScript hybrid code
// returns a DOM root object
function fetch(url) {
var XHR = WSH.CreateObject("Microsoft.XMLHTTP"),
DOM = WSH.CreateObject('htmlfile');
XHR.open("GET",url,true);
XHR.setRequestHeader('User-Agent','XMLHTTP/1.0');
XHR.send('');
while (XHR.readyState!=4) {WSH.Sleep(25)};
DOM.write('<meta http-equiv="x-ua-compatible" content="IE=9" />');
DOM.write(XHR.responseText);
return DOM;
}
var DOM = fetch(WSH.Arguments(0)),
links = DOM.getElementsByTagName('a');
for (var i in links)
if (links[i].href && /\/post\/view\//i.test(links[i].href))
WSH.Echo(links[i].href);
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
