How to retrieve HTML page in proper encoding using Java?

Question

How can I read HTTP stream with HTML page in page's encoding?

Here is a code fragment I use to get the HTTP stream. InputStreamReader has the encoding optional argument, but I have no ideas about the way to obtain it.

URLConnection conn = url.openConnection();
InputStream is = conn.getInputStream();
BufferedReader d = new BufferedReader(new InputStreamReader(is));

Answer 1

Retrieving a Webpage is a reasonably complicated process. That's why libraries such as HttpClient exist. My advice is that unless you have a really compelling reason otherwise, use HttpClient.

Answer 2

When the connection is establised thru

URLConnection conn = url.openConnection();

you can get the encoding method name thru url.getContentEncoding() so pass this String to InputStreamReader() so the code looks like

BufferedReader d = new BufferedReader(new InputStreamReader(is,url.getContentEncoding()));