How to match the color models of BufferedImage and Mat?

Question

I'm trying to convert BufferedImage to Mat for a large set of images with different file types downloaded from the internet. Because I am scraping the images from websites, I have no control over the file formats. I can easily load them to BufferedImage without knowing the format, but to convert them to Mat, I need to know the image type. Unfortunately, there doesn't seem to be a nice correspondence between CvType and BufferedImage types.

CvType represents image types with the format CV_<bit-depth>{U|S|F}C<number_of_channels> where U is unsigned char, S is signed char, and F is float.

BufferedImage types have more variety in the representation including a symmetrical channels (TYPE_4BYTE_ABGR), varying numbers of bits (TYPE_BYTE_BINARY), and whatever an indexed byte image is (TYPE_BYTE_INDEXED).

Based on the documentation, I tried to complete my own correspondence.

BufferedImage imgBuffer = ImageIO.read(new File("example.gif"));

//Save file as reference
File outputfile = new File("temp/image.png");
ImageIO.write(imgBuffer, "png", outputfile);

//My correspondance
int curCVtype = -1;
switch (imgBuffer.getType()) {
case BufferedImage.TYPE_3BYTE_BGR:
    curCVtype = CvType.CV_8UC3;
    break;
case BufferedImage.TYPE_BYTE_GRAY:
    curCVtype = CvType.CV_8UC1;
    break;
case BufferedImage.TYPE_INT_BGR:
case BufferedImage.TYPE_INT_RGB:
    curCVtype = CvType.CV_8SC3;
    break;
case BufferedImage.TYPE_INT_ARGB:
case BufferedImage.TYPE_INT_ARGB_PRE:
    curCVtype = CvType.CV_8SC4;
    break;
default:
//  The types not handled by my correspondence
//                  BufferedImage.TYPE_BYTE_BINARY;
//                  BufferedImage.TYPE_USHORT_GRAY;
//                  BufferedImage.TYPE_4BYTE_ABGR;
//                  BufferedImage.TYPE_4BYTE_ABGR_PRE;
//                  BufferedImage.TYPE_BYTE_INDEXED;
//                  BufferedImage.TYPE_CUSTOM;

    System.out.println("Unsupported format:" + imgBuffer.getType());
    //Here I choose a default type
    curCVtype = CvType.CV_8SC3;
}

//Convert to Mat
byte[] pixels = ((DataBufferByte) imgBuffer.getRaster().getDataBuffer()).getData();
Mat img = new Mat(imgBuffer.getHeight(), imgBuffer.getWidth(), curCVtype);
img.put(0, 0, pixels);

//Write the output to compare
Imgcodecs.imwrite("temp/image_mat.png", img);

Questions

1. Am I going about this correctly? Or is there a better approach?
2. What are the correct correspondences for the types that are in my default case?

Examples

Input

Converted to PNG by BufferedImage

Output PNG after conversion to Mat. BufferedImage type wasTYPE_BYTE_INDEXED converted to CvType.CV_8SC3

Output PNG after conversion to Mat. BufferedImage type wasTYPE_BYTE_INDEXED converted to CvType.CV_8UC3

Resources:

My starting code came from Converting BufferedImage to Mat in opencv .

What I know about CvTypes came from What's the difference between cvtype values in OPENCV? .

Answer 1

Thanks to Miki and haraldK for the helpful comments.

My solution for unknown image types retrieves the pixels in RGB format, and put them into a Mat of CvType.CV_8UC4 . Finally, reorder the channels using Core.mixChannels to the OpenCV prefered order: BGR(A).

This example only reorders the channels for the unknown image types, but all non BGR image types would need to be reordered.

BufferedImage imgBuffer = ImageIO.read(new File("example.gif"));

//Save image as reference
File outputfile = new File("temp/image.png");
ImageIO.write(imgBuffer, "png", outputfile);

//My correspondance

int curCVtype = CvType.CV_8UC4; //Default type
boolean supportedType = true;

switch (imgBuffer.getType()) {
case BufferedImage.TYPE_3BYTE_BGR:
    curCVtype = CvType.CV_8UC3;
    break;
case BufferedImage.TYPE_BYTE_GRAY:
case BufferedImage.TYPE_BYTE_BINARY:
    curCVtype = CvType.CV_8UC1;
    break;
case BufferedImage.TYPE_INT_BGR:
case BufferedImage.TYPE_INT_RGB:
    curCVtype = CvType.CV_32SC3;
    break;
case BufferedImage.TYPE_INT_ARGB:
case BufferedImage.TYPE_INT_ARGB_PRE:
    curCVtype = CvType.CV_32SC4;
    break;
case BufferedImage.TYPE_USHORT_GRAY:
    curCVtype = CvType.CV_16UC1;
    break;
case BufferedImage.TYPE_4BYTE_ABGR:
case BufferedImage.TYPE_4BYTE_ABGR_PRE:
    curCVtype = CvType.CV_8UC4;
    break;
default:
    // BufferedImage.TYPE_BYTE_INDEXED;
    // BufferedImage.TYPE_CUSTOM;
    System.out.println("Unsupported format:" + imgBuffer.getType());
    supportedType = false;
}

//Convert to Mat
Mat img = new Mat(imgBuffer.getHeight(), imgBuffer.getWidth(), curCVtype);
if (supportedType) {
    // Insert pixel buffer directly
    byte[] pixels = ((DataBufferByte) imgBuffer.getRaster().getDataBuffer()).getData();
    img.put(0, 0, pixels);
} else {
    // Convert to RGB first
    int height = imgBuffer.getHeight();
    int width = imgBuffer.getWidth();
    int[] pixels = imgBuffer.getRGB(0, 0, width - 1, height - 1, null, 0, width);

    // Convert ints to bytes
    ByteBuffer byteBuffer = ByteBuffer.allocate(pixels.length * 4);
    IntBuffer intBuffer = byteBuffer.asIntBuffer();
    intBuffer.put(pixels);

    byte[] pixelBytes = byteBuffer.array();

    img.put(0, 0, pixelBytes);

    // Reorder the channels for Opencv BGRA format from
    // BufferedImage ARGB format
    Mat imgMix = img.clone();
    ArrayList<Mat> imgSrc = new ArrayList<Mat>();
    imgSrc.add(imgMix);

    ArrayList<Mat> imgDest = new ArrayList<Mat>();
    imgDest.add(img);

    int[] fromTo = { 0, 3, 1, 2, 2, 1, 3, 0 }; //Each pair is a channel swap
    Core.mixChannels(imgSrc, imgDest, new MatOfInt(fromTo));
}

//Save output image
Imgcodecs.imwrite("temp/image_mat.png", img);

The new output image