Fast loading and drawing of RGB data in BufferedImage

Question

In some Java code running on Windows, I'm reading some large blocks of RGB data from disk and want to display this to screen as quickly as possible. The RGB data is 8 bits per channel without any alpha. Currently I have code like the following to create the BufferedImage.

BufferedImage getBufferedImage(File file, int width, int height) {

    byte[] rgbData = readRGBFromFile(file);

    WritableRaster raster = Raster.createInterleavedRaster(
        rgbData, width, height, 
        width * 3, // scanlineStride
        3, // pixelStride
        new int[]{0, 1, 2}, // bandOffsets
        null);

    ColorModel colorModel = new ComponentColorModel(
        ColorSpace.getInstance(ColorSpace.CS_sRGB), 
        new int[]{8, 8, 8}, // bits
        false, // hasAlpha
        false, // isPreMultiplied
        ComponentColorModel.OPAQUE, 
        DataBuffer.TYPE_BYTE);

    return new BufferedImage(colorModel, raster, false, null);
}

The problem is that the performance of rendering this to the screen is pretty slow. Around 250 - 300 ms. I've read that for the best performance you need to display in a BufferedImage that's compatible with the screen. To do that, I pass the buffered image returned from the above method to a method like this.

BufferedImage createCompatibleImage(BufferedImage image)
{
    GraphicsConfiguration gc = GraphicsEnvironment.
        getLocalGraphicsEnvironment().
        getDefaultScreenDevice().
        getDefaultConfiguration();

    BufferedImage newImage = gc.createCompatibleImage(
        image.getWidth(), 
        image.getHeight(), 
        Transparency.TRANSLUCENT);

    Graphics2D g = newImage.createGraphics();
    g.drawImage(image, 0, 0, null);
    g.dispose();

    return newImage;
}

That method essentially converts it from RGB to ARGB on Windows and it really speeds up the displaying, but this method takes ~300 ms for a 1600 x 1200 RGB data block. So now I've basically traded the performance hit of the drawing problem to a converting problem.

300ms is about the same time as it takes to load the RGB data from disk. I would think I could do something faster.

Is there a better way I can do the conversion? Or would it help if I modified the RGB data and added an alpha channel myself beforehand? If so what would my Raster and ColorModel look like. Also, since my RGB data doesn't contain transparency can I get any performance improvements by using pre multiplied alpha or something?

Sorry, bit I'm a little lost on this ColorModel, Raster stuff.

Thanks!

Answer 1

I realize this is a really old question, I'm just posting this for anybody else who might stumble upon this question looking for more options. I had an issue recently where I was attempting to take a large (720p) RGB byte[] and render it to a BufferedImage. The original implementation I was using looked something like this (simplified here):

public void processFrame(byte[] frame, int width, int height)
{
   DataBuffer videoBuffer = new DataBufferByte(frame,frame.length);
   BufferedImage currentImage = new BufferedImage(width,height,BufferedImage.TYPE_3BYTE_BGR);
   ComponentSampleModel sampleModel = new ComponentSampleModel(DataBuffer.TYPE_BYTE,width,height,3,width*3,new int[] {2,1,0});
   Raster raster = Raster.createRaster(sampleModel,videoBuffer,null);
   currentImage.setData(raster);
}

Even with optimizations like creating the BufferedImage and ComponentSampleModel once and reusing them, the final step of calling setData on the BufferedImage was still taking on the order of 50-60 milliseconds, which is unacceptable.

What I ended up realizing is that, at least for my scenario, you can actually write to the backing byte array of the BufferedImage directly and bypass most of the intermediate processing (assuming the backing metadata for the image is already correct). So I changed my code to look like this:

public void processFrame(byte[] frame, int width, int height)
{
   BufferedImage currentImage = new BufferedImage(width,height,BufferedImage.TYPE_3BYTE_BGR);
   byte[] imgData = ((DataBufferByte)currentImage.getRaster().getDataBuffer()).getData();
   System.arraycopy(frame,0,imgData,0,frame.length);
}

Just by doing this, my performance improved by about a factor of 20. I now process the same frames in 3-5 milliseconds instead of 50-60 milliseconds.

This may not be applicable for all cases, but I thought I'd share in case someone else finds it useful.

Answer 2

After playing around with this I have a decent answer that works for Windows if the current graphics configuration is using ARGB integer packed rasters.

What I do is create the compatible BufferedImage first, then I manually convert my RGB bytes array to an ARGB int array. Then I get the Raster from the compatible BufferedImage and write my ARGB ints into it. This is much faster.

I also have a class that checks if the compatible BufferedImage is in the format I expect, if it isn't it defaults to the older slower approach.

Here is the class. Hope it helps you.

/**
 * This class can read chunks of RGB image data out of a file and return a BufferedImage.
 * It may use an optimized technique for loading images that relies on assumptions about the 
 * default image format on Windows.
 */
public class RGBImageLoader
{
    private byte[] tempBuffer_;
    private boolean fastLoading_;

    public RGBImageLoader()
    {
        fastLoading_ = canUseFastLoadingTechnique();
    }

    private boolean canUseFastLoadingTechnique()
    {
        // Create an image that's compatible with the screen
        GraphicsConfiguration gc = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice().getDefaultConfiguration();
        BufferedImage image = gc.createCompatibleImage(100, 100, Transparency.TRANSLUCENT);

        // On windows this should be an ARGB integer packed raster. If it is then we can 
        // use our optimization technique

        if(image.getType() != BufferedImage.TYPE_INT_ARGB)
            return false;

        WritableRaster raster = image.getRaster();

        if(!(raster instanceof IntegerInterleavedRaster))
            return false;

        if(!(raster.getDataBuffer() instanceof DataBufferInt))
            return false;

        if(!(image.getColorModel() instanceof DirectColorModel))
            return false;

        DirectColorModel colorModel = (DirectColorModel) image.getColorModel();

        if(!(colorModel.getColorSpace() instanceof ICC_ColorSpace) ||
             colorModel.getNumComponents() != 4 ||
             colorModel.getAlphaMask() != 0xff000000 ||
             colorModel.getRedMask() != 0xff0000 ||
             colorModel.getGreenMask() != 0xff00 ||
             colorModel.getBlueMask() != 0xff)
            return false;

        if(raster.getNumBands() != 4 ||
           raster.getNumDataElements() != 1 ||
           !(raster.getSampleModel() instanceof SinglePixelPackedSampleModel))
            return false;

        return true;
    }

    public BufferedImage loadImage(File file, int width, int height, long imageOffset) throws IOException
    {
        if(fastLoading_)
            return loadImageUsingFastTechnique(file, width, height, imageOffset);
        else
            return loadImageUsingCompatibleTechnique(file, width, height, imageOffset);
    }

    private BufferedImage loadImageUsingFastTechnique(File file, int width, int height, long imageOffset) throws IOException
    {
        int sizeBytes = width * height * 3;

        // Make sure buffer is big enough
        if(tempBuffer_ == null || tempBuffer_.length < sizeBytes)
            tempBuffer_ = new byte[sizeBytes];

        RandomAccessFile raf = null;
        try
        {
            raf = new RandomAccessFile(file, "r");

            raf.seek(imageOffset);

            int bytesRead = raf.read(tempBuffer_, 0, sizeBytes);
            if (bytesRead != sizeBytes)
                throw new IOException("Invalid byte count. Should be " + sizeBytes + " not " + bytesRead);

            GraphicsConfiguration gc = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice().getDefaultConfiguration();
            BufferedImage image = gc.createCompatibleImage(width, height, Transparency.TRANSLUCENT);
            WritableRaster raster = image.getRaster();
            DataBufferInt dataBuffer = (DataBufferInt) raster.getDataBuffer();

            addAlphaChannel(tempBuffer_, sizeBytes, dataBuffer.getData());

            return image;
        }
        finally
        {
            try
            {
                if(raf != null)
                raf.close();
            }
            catch(Exception ex)
            {
            }
        }
    }

    private BufferedImage loadImageUsingCompatibleTechnique(File file, int width, int height, long imageOffset) throws IOException
    {
        int sizeBytes = width * height * 3;

        RandomAccessFile raf = null;
        try
        {
            raf = new RandomAccessFile(file, "r");

            // Lets navigate to the offset
            raf.seek(imageOffset);

            DataBufferByte dataBuffer = new DataBufferByte(sizeBytes);
            byte[] bytes = dataBuffer.getData();

            int bytesRead = raf.read(bytes, 0, sizeBytes);
            if (bytesRead != sizeBytes)
                throw new IOException("Invalid byte count. Should be " + sizeBytes + " not " + bytesRead);

            WritableRaster raster = Raster.createInterleavedRaster(dataBuffer, // dataBuffer
                            width, // width
                            height, // height
                            width * 3, // scanlineStride
                            3, // pixelStride
                            new int[]{0, 1, 2}, // bandOffsets
                            null); // location

            ColorModel colorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB), // ColorSpace
                            new int[]{8, 8, 8}, // bits
                            false, // hasAlpha
                            false, // isPreMultiplied
                            ComponentColorModel.OPAQUE, DataBuffer.TYPE_BYTE);

            BufferedImage loadImage = new BufferedImage(colorModel, raster, false, null);

            // Convert it into a buffered image that's compatible with the current screen.
            // Not ideal creating this image twice....
            BufferedImage image = createCompatibleImage(loadImage);

            return image;
        }
        finally
        {
            try
            {
                if(raf != null)
                raf.close();
            }
            catch(Exception ex)
            {
            }
        }
    }

    private BufferedImage createCompatibleImage(BufferedImage image)
    {
        GraphicsConfiguration gc = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice().getDefaultConfiguration();

        BufferedImage newImage = gc.createCompatibleImage(image.getWidth(), image.getHeight(), Transparency.TRANSLUCENT);

        Graphics2D g = newImage.createGraphics();
        g.drawImage(image, 0, 0, null);
        g.dispose();

        return newImage;
    }


    private void addAlphaChannel(byte[] rgbBytes, int bytesLen, int[] argbInts)
    {
        for(int i=0, j=0; i<bytesLen; i+=3, j++)
        {
            argbInts[j] = ((byte) 0xff) << 24 |                 // Alpha
                        (rgbBytes[i] << 16) & (0xff0000) |      // Red
                        (rgbBytes[i+1] << 8) & (0xff00) |       // Green
                        (rgbBytes[i+2]) & (0xff);               // Blue
        }
    }

}