Extract frames from a video in real-time android

Question

I'm doing an app to record a video. What I would like is that when the app is recording the video, each frame is also saved in an arraylist of RGB values in order to extract particular information from it. I know that the two processes (video recording and extraction of frames) are asynchronous, but this is not a problem: the process of extraction can finish after the video recording.

Can someone tell me how I can extract the frames from the video?

Thanks very much.

Answer 1

Using the camera object you can override a function setPreviewCallback. from that function you would get an array of byte data. Which you can convert to bitmap or you can save them in an array of array.

Assuming that you were extends SurfaceView and Implements SurfaceHolder.Callback

Code looks like,

mCamera.setPreviewCallback(new PreviewCallback() {
    public void onPreviewFrame(byte[] data, Camera camera) {

        //do some operations using data

    }
});

If you want to get the RGB value you can use this function,

static public void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width,
            int height) {
        final int frameSize = width * height;

        for (int j = 0, yp = 0; j < height; j++) {
            int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
            for (int i = 0; i < width; i++, yp++) {
                int y = (0xff & ((int) yuv420sp[yp])) - 16;
                if (y < 0)
                    y = 0;
                if ((i & 1) == 0) {
                    v = (0xff & yuv420sp[uvp++]) - 128;
                    u = (0xff & yuv420sp[uvp++]) - 128;
                }

                int y1192 = 1192 * y;
                int r = (y1192 + 1634 * v);
                int g = (y1192 - 833 * v - 400 * u);
                int b = (y1192 + 2066 * u);

                if (r < 0)
                    r = 0;
                else if (r > 262143)
                    r = 262143;
                if (g < 0)
                    g = 0;
                else if (g > 262143)
                    g = 262143;
                if (b < 0)
                    b = 0;
                else if (b > 262143)
                    b = 262143;

                rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000)
                        | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
            }
        }
    }

I think this is what you want as an answer. If i am wrong, please let me know whats is your actual requirement

For getting the green value, you can use the below code,

static public void calculateColor(int[] rgb, int[] color,
            int width, int height, int component) {
        for (int bin = 0; bin < 256; bin++) {
            color[bin] = 0;
        } // bin

        int start = 0;
        int end = width * height;

        if (component == 0) // red
        {
            for (int pix = start; pix < end; pix += 3) {
                int pixVal = (rgb[pix] >> 16) & 0xff;
                color[pixVal]++;
            } // pix
        } else if (component == 1) // green
        {
            for (int pix = start; pix < end; pix += 3) {
                int pixVal = (rgb[pix] >> 8) & 0xff;
                color[pixVal]++;
            } // pix
        } else // blue
        {
            for (int pix = start; pix < end; pix += 3) {
                int pixVal = rgb[pix] & 0xff;
                color[pixVal]++;
            } // pix
        }
    }

Calling this function would return the value of green for every pixel. If you wants the green value for a particular pixel, you have to modify this code.