Socket closing problem - last portion of data is lost

Question

I have a Java program that accepts connections, receives HTTP requests and sends HTTP replies and some data stored in file (this is a part of caching proxy). Removing everything irrelevant, my code looks like this:

FileInputStream fileInputStream = new FileInputStream(file);
OutputStream outputStream = socket.getOutputStream();
byte[] buf = new byte[BUFFER_SIZE];
int len = 0;
while ((len = fileInputStream.read(buf)) > 0) {
    outputStream.write(buf, 0, len);
}
outputStream.flush();
socket.close();

This code is executed in particular thread for each connected client.

When I deal with small files (.htm, .gif, .swf, etc.), everything works fine (however, I don't see anything wrong in browser). But when I download large files (.iso), especially several files simultaneously, when system is under load, sometimes I get really strange behavior. Browser downloads 99.99% of a file and when there are less than BUFFER_SIZE of undownloaded bytes, downloading stops for a few seconds and then browser says that error has occured. I can not understand what happens, because all data is successfully read and even all data is successfully written to outputStream. As you can see, I even do flush(), but it takes no result.

Can anyone explain me what happens?

EDIT
Uploaded project to filehosting.org.
Download source files. There is zip archive with source code, Build.xml and Readme.txt. Use ant to build solution. Described problem occurs in ClientManager.java, you'll find a comment there.

Answer 1

Based on a quick trawl through the JDK 1.6 codebase:

• socket.getOutputStream() returns a SocketOutputStream instance
• flush() indeed has no effect on a SocketOutputStream instance
• write() on a SocketOutputStream instance does not seem to buffer anything in Java code
• shutdownOutput() should ensure that any outstanding data is written before shutting down the output side of the socket. At least, the comments say that.

However, some of the Socket etc implementation is native methods, and I didn't delve into that.

Based on what I could tell, the "correct" sequence would be:

socket.shutdownOutput();
socket.close();

However, you say you've tried that. Is it possible that the application at the other end is closing the TCP/IP connection early?

Another thought: you tried setSoLinger(true, 60000), but 60000 seconds is possibly longer that the OS allows. Try setSoLinger(true, 60), and try doing it before you open the output stream.

Answer 2

Not sure why that would happen, I don't see a problem with your code. Have you tried using a BufferedInputStream to enclose the FileInputStream?