Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Mask jsoup as a Browser when downloading html

Tags:

jsoup

is it possible to mask

Jsoup.connect("http://xyz.com").get().html();

as a browser call to the website?

I try to build a wallpaper download tool and experiencing problems when downloading the page form the server.

If I download the page, say with Firefox, uploading this to a server of mine it is downloaded properly...

I figured it must be some browser based script prevention which doesn't let you parse the page automatically...

like image 302
Ray Avatar asked Feb 24 '23 09:02

Ray


1 Answers

Try setting the User-Agent header on the request. Ex:

String ua = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.122 Safari/534.30";
Jsoup.connect("http://example.com").userAgent(ua).get().html();
like image 157
Matt Ball Avatar answered Apr 15 '23 17:04

Matt Ball