Android -- analyze encoded frames when while video

Question

I have a camera app that analyzes video when recording (detect camera movement). I use camera2 API and I get frames (from camera) when recording video (with MediaRecorder). But the video file should be analyzed once more on a server. The problem is that I analyze non-compressed frames on phone. But server analyzes video frames after compression and decompression. And those frames are definitelly different.

My MotionDetector takes frames from camera, calculates shift between current frame and previous frame using Phase Correlate. After video file uploaded to the server, it is analyzed with the same MotionDetector.

If I compare shift-between-frames on uncomperred frames to shift-between-frames on compressed frames -- they are obviously different.

Is there a way to get frames after compression?

Currently data flow looks like:

1. Frames from camera
2.1 frames from camera displayed on a screen
2.2 frames from camera are fed to MediaRecorder.
2.3 frames from camera are fed to my MotionDetector
3. MediaRecorder encodes camera frames into H264 format.
4 MediaRecorder output stored to file

I need the data workflow to look like:

1. Frames from camera
2.1 frames from camera displayed on a screen
2.2 frames from camera are fed into MediaRecorder.
3. MediaRecorder encodes camera frames into H264 format.
4.1 MediaRecorder output stored to file
4.2 MediaRecorder output goes to MediaCodec
5. MediaCodec decodes frames.
6. frames from MediaCoded are fed to MotionDetector.

Is there a way to do that?

Answer 1

If you stop using MediaRecorder, and just use MediaCodec directly for encoding video, you should be able to inspect the encoded video buffers before writing them to a MediaMuxer for storage to disk.

That's a lot more complicated than just using MediaRecorder directly, but a lot more flexible for cases like this.