Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reading a web page in Java IOException Premature EOF

I am frequently getting a 'Premature EOF' Exception when reading a web page.

The following is the StackTrace

java.io.IOException: Premature EOF
    at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:556)
    at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:600)
    at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:687)
    at java.io.FilterInputStream.read(FilterInputStream.java:133)
    at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2968)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.BufferedReader.fill(BufferedReader.java:154)
    at java.io.BufferedReader.readLine(BufferedReader.java:317)
    at java.io.BufferedReader.readLine(BufferedReader.java:382)
    at Utilities.getPage(Utilities.java:24)  while ((line = rd.readLine()) != null) {
    at TalkPage.<init>(TalkPage.java:15)
    at Updater.run(Updater.java:65)

Following is the getPage() method

public static String getPage(String urlString) throws Exception {
    URL url = new URL(urlString);
    URLConnection conn = url.openConnection();
    BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
    StringBuffer sb = new StringBuffer();
    String line;
    while ((line = rd.readLine()) != null) {  // LINE 24
        sb.append(line);
    }
    return sb.toString();
}

What is a permature EOFException and why is it occuring in this particular case and how can it be avoided?

Some other information: The size of the page being read is around 20 KB and I'm reading many such pages in my program ( around 20 000 )

like image 350
Ranjith Avatar asked Nov 03 '12 14:11

Ranjith


2 Answers

This may be because you are reading the content line by line and for the last line the file may be missing a return, to signal the end of line. Replace your while with this:

int BUFFER_SIZE=1024;
char[] buffer = new char[BUFFER_SIZE]; // or some other size, 
int charsRead = 0;
while ( (charsRead  = rd.read(buffer, 0, BUFFER_SIZE)) != -1) {
  sb.append(buffer, 0, charsRead);
}
like image 135
dan Avatar answered Oct 23 '22 01:10

dan


This could be because the server is closing the connection. I have experienced the exact same issue when I had a piece of code which opened a connection, did some other processing, and only then tried to download the contents of the input stream - by the time it to the stream after spending a few seconds on other processing, the server had apparently closed the connection, resulting in IOException: Premature EOF. The solution was to be careful to always immediately handle the contents of the stream - otherwise, you are leaving an HTTP connection open and idle, and eventually the server on the other end of the line will hang up on you.

like image 28
chackley Avatar answered Oct 23 '22 00:10

chackley