I know that HtmlUnit
simulates a browser, while HttpClient
doesn't.
In HtmlUnit
, when a page is loaded and there is a JavaScript inside, will the script be executed? If the script sets a cookie, will the cookie set in HtmlUnit
's browser and accessible from Java code?
Is there anything that can be done using HttpClient
, but not using HtmlUnit
? In HtmlUnit
, can we start with a POST request and modify any part of HTTP request including method, URI, HTTP version, headers, and body?
What are the advantages of HttpClient
over HtmlUnit
?
HttpClient
is a library at a lower-level, to send HTTP requests and retrieve responses.
HtmlUnit
is at a higher level, and internally uses HttpClient
to make HTTP requests, but also handles JavaScript (through Rhino
and internal DOM implementation), XPath (through Xalan
), CSS (through CSSParser
), malformed HTML (through NekoHtml
), WebSockets (through Jetty
), etc.
You can modify the outgoing requests and response in HtmlUnit
by something like:
new WebConnectionWrapper(webClient) {
public WebResponse getResponse(WebRequest request) throws IOException {
WebResponse response = super.getResponse(request);
if (request.getUrl().toExternalForm().contains("my_url")) {
String content = response.getContentAsString("UTF-8");
//change content
WebResponseData data = new WebResponseData(content.getBytes("UTF-8"),
response.getStatusCode(), response.getStatusMessage(), response.getResponseHeaders());
response = new WebResponse(data, request, response.getLoadTime());
}
return response;
}
};
as hinted here.
You can change the used HttpClient
in HtmlUnit
by overriding HttpWebConnection.createHttpClient()
.
You can make POST
request by:
WebRequest webRequest = new WebRequest(url, HttpMethod.POST);
HtmlPage page = webClient.getPage(webRequest);
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With