How to use LUT with JavaScript?

Question

I'm new to image processing.

I want to use JavaScript to apply effects to images using LUTs (LookUp Tables) or the corresponding lookup PNGs, something like this:

I have googled a lot and couldn't find an article or any resource which will describe the exact process of pixel transformation using LUTs.

I have found a nice article here, which describes the 1D and 3D LUTs and the differences between them. But its still is not fully clear for me.

I want something like this, which is done for iOS.

P.S. Please do not post links/answers regarding image filtering libs, which are using convolution matrices for effects or for filters.

Update:

Finally! I have got the answer thanks to @abbath. I've created a gist in GitHub, which you can find here.

Answer 1

I am also new to the topic, but I hope it helps what I've found out so far. Your image (the LUT) is a representation of a 3D array, imagine a 64x64x64 cube. This representation is in 2D, one plane is a 64x64 square. You need 64 piece of this plane, which is why the png has 8x8 squares in it. If you look carefully, the upperleft square has red (R of RGB) on the X axis and green (G) on the Y axis. The blue (B) is the Z axis (I imagine it upwards), which can't be representated in 2D, so it comes on the next 64x64 squares. On the last (lower right) square you can see that it is mostly blue, where the red and green coordinates are 0.

So the next question is how to project an image with this LUT. I have tried it out in Java, in my opinion you will have to lose some information because 64x64x64 is far less than 256x256x256, in this case, you have to divide by 4 each pixel color value.

The steps:

1. Iterate through the pixels of your original image. In one iteration you will have one pixel of the original image. Get the R, G, B values of that pixel.
2. Divide the values by four, so you will have a value less than 64.
3. Get the corresponding pixel from your LUT, as if it was a 64x64x64 cube. So if RGB=(255,0,255) was on the original pixel, than divide it to get (63,0,63), then check the B value, to get the corresponding square on the png, which will be the lower right one (in the case of b=63), and get the r,g=(63,0) pixel of it, which purpleish on your png image.

I checked now on android, with java code, its fast enough in my opinion. Below is the code I've written, I hope it's enough to port it to javascript. (I hope the comments are enough to understand)

public Bitmap applyEffectToBitmap(Bitmap src, Bitmap lutbitmap) {
    //android specific code, storing the LUT image's pixels in an int array
    int lutWidth = lutBitmap.getWidth();
    int lutColors[] = new int[lutWidth * lutBitmap.getHeight()];
    lutBitmap.getPixels(lutColors, 0, lutWidth, 0, 0, lutWidth, lutBitmap.getHeight());

    //android specific code, storing the source image's pixels in an int array
    int srcWidth = src.getWidth();
    int srcHeight = src.getHeight();
    int[] pix = new int[srcWidth * srcHeight];

    src.getPixels(pix, 0, srcWidth, 0, 0, srcWidth, srcHeight);

    int R, G, B;
    //now we can iterate through the pixels of the source image
    for (int y = 0; y < srcHeight; y++){
        for (int x = 0; x < srcWidth; x++) {
            //index: because the pix[] is one dimensional
            int index = y * srcWidth+ x;
            //pix[index] returns a color, we need it's r g b values, thats why the shift operator is used
            int r = ((pix[index] >> 16) & 0xff) / 4;
            int g = ((pix[index] >> 8) & 0xff) / 4;
            int b = (pix[index] & 0xff) / 4;

            //the magic happens here: the 3rd step above: blue pixel describes the 
            //column and row of which square you'll need the pixel from
            //then simply add the r and green value to get the coordinates 
            int lutX = (b % 8) * 64 + r;
            int lutY = (b / 8) * 64 + g;
            int lutIndex = lutY * lutWidth + lutX;


            //same pixel getting as above, but now from the lutColors int array
            R = ((lutColors[lutIndex] >> 16) & 0xff);
            G = ((lutColors[lutIndex] >> 8) & 0xff);
            B = ((lutColors[lutIndex]) & 0xff);


            //overwrite pix array with the filtered values, alpha is 256 in my case, but you can use the original alpha
            pix[index] = 0xff000000 | (R << 16) | (G << 8) | B;
        }
    }
    //at the end of operations pix[] has the transformed pixels sou can set them to your new bitmap
    //some android specific code is here for creating bitmaps
    Bitmap result = Bitmap.createBitmap(srcWidth, srcHeight, src.getConfig());
    result.setPixels(pix, 0, srcWidth, 0, 0, srcWidth, srcHeight);
    return result ;
}

Now I've successfully implemented in javascript too, check for using the Math.floor() functions:

for (var i=0;i<imgData.data.length;i+=4){
    var r=Math.floor(imgData.data[i]/4);
    var g=Math.floor(imgData.data[i+1]/4);
    var b=Math.floor(imgData.data[i+2]/4);    
    var a=255;


    var lutX = (b % 8) * 64 + r;
    var lutY = Math.floor(b / 8) * 64 + g;
    var lutIndex = (lutY * lutWidth + lutX)*4;

    var Rr =  filterData.data[lutIndex];
    var Gg =  filterData.data[lutIndex+1];    
    var Bb =  filterData.data[lutIndex+2];

    imgData.data[i] = Rr;
    imgData.data[i+1] = Gg;
    imgData.data[i+2] = Bb;
    imgData.data[i+3] = 255;

}

Check it here: http://jsfiddle.net/gxu080ve/1/ The lut image is in byte code, sorry for that.

This code only applies for 64x64x64 3DLUT images. The parameters vary if your LUT has other dimensions; the / 4, * 64, % 8, / 8 must be changed for other dimensions, but in this question's scope the LUT is 64x64x64.

Answer 2

I'm trying to do something similar in c++. 3D LUTs have 3 columns of values usually between 0 and 1. The columns are r g b respectively and they represent a mapping of values. If you open up one of these files of size nxnxn, you would read each value into it's respective R array or G array or B array. Then to recreate the mapping internally, you would iterate and fill up rgb maps with double/float keys and values

for every b
    for every g
        for every r
            // r g and b are indices
            index = b * n^2 + g * n + r
            Rmap.push((r + 1)/n, Rarray[index])
            // Same for g and b
            Gmap.push((g + 1)/n, Garray[index]
            Bmap.push((b + 1)/n, Barray[index]

Once you have your mappings, you can apply the lookup to every pixel in an image. Say the image had Max color 255, so you take the (r,g,b)/255 of each pixel and apply the lookup then convert back by *255. As far as I know, the convention to calculate values you can't lookup, is to perform trilinear interpolation. I haven't had too much success yet, but this is my two cents.