Send a stream of images using ImageIO?

Question

I have a ServerSocket and a Socket set up so the ServerSocket sends a stream of images using ImageIO.write(....) and the Socket tries to read them and update a JFrame with them. So I wondered if ImageIO could detect the end of an image. (I have absolutely no knowledge of the JPEG format, so I tested it instead)

Apparently, not.

On the server side, I sent images continuously by using ImageIO.write(...) in loop with some sleeping in between. On the client side, ImageIO read the first image no problem, but on the next one it returned null. This is confusing. I was expecting it to either block on reading the first image (because it thinks the next image is still part of the same image), or succeed at reading all of them (because it works). What is going on? It looks like ImageIO detects the end of the first image, but not the second one. (The images, by the way, are similar to each other roughly) Is there an easy way to stream images like this or do I have to make my own mechanism that reads the bytes into a buffer until it reaches a specified byte or sequence of bytes, at which point it reads the image out of the buffer?

This is the useful part of my server code:

        while(true){
            Socket sock=s.accept();
            System.out.println("Connection");
            OutputStream out=sock.getOutputStream();
            while(!socket.isClosed()){
                BufferedImage img=//get image
                ImageIO.write(img, "jpg", out);
                Thread.sleep(100);
            }
            System.out.println("Closed");
        }

And my client code:

        Socket s=new Socket(InetAddress.getByName("localhost"), 1998);
        InputStream in=s.getInputStream();
        while(!s.isClosed()){
            BufferedImage img=ImageIO.read(in);
            if(img==null)//this is what happens on the SECOND image
            else // do something useful with the image
        }

Answer 1

ImageIO.read(InputStream) creates an ImageInputStream and calls read(ImageInputStream) internally. That latter method is documented to close the stream when it's done reading the image.

So, in theory, you can just get the ImageReader, create an ImageInputStream yourself, and have the ImageReader read from the ImageInputStream repeatedly.

Except, it appears an ImageInputStream is designed to work with one and only one image (which may or may not contain multiple frames). If you call ImageReader.read(0) more than once, it will rewind to the beginning of the (cached) stream data each time, giving you the same image over and over. ImageReader.read(1) will look for a second frame in a multi-frame image, which of course makes no sense with a JPEG.

So, maybe we can create an ImageInputStream, have the ImageReader read from it, and then create a new ImageInputStream to handle subsequent image data in the stream, right? Except, it appears ImageInputStream does all sorts of caching, read-ahead and pushback, which makes it quite difficult to know the read position of the wrapped InputStream. The next ImageInputStream will start reading data from somewhere, but it's not at the end of the first image's data like we would expect.

The only way to be certain of your underlying stream's position is with mark and reset. Since images can be large, you'll probably need a BufferedInputStream to allow a large readLimit.

This worked for me:

private static final int MAX_IMAGE_SIZE = 50 * 1024 * 1024;

static void readImages(InputStream stream)
throws IOException {
    stream = new BufferedInputStream(stream);

    while (true) {
        stream.mark(MAX_IMAGE_SIZE);

        ImageInputStream imgStream =
            ImageIO.createImageInputStream(stream);

        Iterator<ImageReader> i = 
            ImageIO.getImageReaders(imgStream);
        if (!i.hasNext()) {
            logger.log(Level.FINE, "No ImageReaders found, exiting.");
            break;
        }

        ImageReader reader = i.next();
        reader.setInput(imgStream);

        BufferedImage image = reader.read(0);
        if (image == null) {
            logger.log(Level.FINE, "No more images to read, exiting.");
            break;
        }

        logger.log(Level.INFO,
            "Read {0,number}\u00d7{1,number} image",
            new Object[] { image.getWidth(), image.getHeight() });

        long bytesRead = imgStream.getStreamPosition();

        stream.reset();
        stream.skip(bytesRead);
    }
}