Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Jsoup connection with basic access authentication

Tags:

Is there a way in Jsoup to load a document from a website with basic access authentication?

like image 329
user982940 Avatar asked Oct 06 '11 20:10

user982940


People also ask

What is jsoup connect?

jsoup is a Java library for working with real-world HTML. It provides a very convenient API for fetching URLs and extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors. jsoup implements the WHATWG HTML5 specification, and parses HTML to the same DOM as modern browsers do.

What does jsoup parse do?

jsoup can parse HTML files, input streams, URLs, or even strings. It eases data extraction from HTML by offering Document Object Model (DOM) traversal methods and CSS and jQuery-like selectors. jsoup can manipulate the content: the HTML element itself, its attributes, or its text.


2 Answers

With HTTP basic access authentication you need to send the Authorization header along with a value of "Basic " + base64encode("username:password").

E.g.

String username = "foo";
String password = "bar";
String login = username + ":" + password;
String base64login = Base64.getEncoder().encodeToString(login.getBytes());

Document document = Jsoup
    .connect("http://example.com")
    .header("Authorization", "Basic " + base64login)
    .get();

// ...

(explicit specification of character encoding in getBytes() is omitted for brevity as login name and pass is often plain US-ASCII anyway; besides, Base64 always generates US-ASCII bytes)

like image 193
BalusC Avatar answered Oct 15 '22 09:10

BalusC


//Log in
Response res = Jsoup
    .connect("url")
    .data("loginField", "login")
    .data("passwordField", "password")
    .method(Method.POST)
    .execute();

Document doc = res.parse();


//Keep logged in
Map<String, String> cookies = res.cookies();

Document doc2 = Jsoup
    .connect("url")
    .cookies(cookies)
    .get();
like image 45
Igor Brusamolin Lobo Santos Avatar answered Oct 15 '22 08:10

Igor Brusamolin Lobo Santos