making multiple http request efficiently

Question

I want to make a few million http request to web service of the form- htp://(some ip)//{id}

I have the list of ids with me. Simple calculation has shown that my java code will take around 4-5 hours to get the data from the api The code is

URL getUrl = new URL("http url");
URLConnection conn = getUrl.openConnection();
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuffer sbGet = new StringBuffer();
String getline;
while ((getline = rd.readLine()) != null)
{
    sbGet.append(getline);
}
rd.close();
String getResponse = sbGet.toString();

Is there a way to more efficiently make such requests which will take less time

Answer 1

One way is to use an executor service with a fixed thread pool (the size depends how much the target HTTP service can handle) and bombard requests to the service in parallel. The Runnable would basically perform the steps you outlined in your sample code, btw.