Large JPEG/PNG Image Sequence Looping

Question

I have been working on my project about remotely sensed image processing, and image sequence looping. Each resulting image (in JPEG or PNG format) has approximately 8000 * 4000 pixels. Our users usually want to loop an image sequence (more than 50 images) on the basis of region of interest at a time. Thus, I have to extract the required viewing area from the each image according to user's visualization client size. For example, if user's current client view is 640 * 480, I'll have to find a size of 640 * 480 data block from each original image based on the current x (columns) and y (rows) coordinates, and remap to the client view. When user pans to another viewing area by mouse dragging, our program must accordingly re-load regional data out of each original image as soon as possible.

I know neither JPEG library nor PNG library has some built-in data block read routines, such as long ReadRectangle (long x0, long y0, long x1, long y1, char* RectData); long ReadInaRectangle (long x0, long y0, short width, short height, char* RectData);

The built-in JPEG decompressor lacks this kind of functionality. I know that JPEG2000 format has provisions for decompressing a specific area of the image. I'm not entirely sure about JEPG.

Someone suggest that I use CreateFileMapping, MapViewOfFile, and CreateDIBSection to commit the number of bytes of a file mapping to map to the view. Unlike the simple flat binary image formats such *.raw, *.img, and *.bmp, JPEG's Blob will contain not only the image data but also the complicated JPG header. So it's not easy to map a block of data view out of the JPEG file.

Someone recommend that I use image tiling or image pyramid technology to generate sub-images, just like mnay popular, image visualization (Google Earth, and etc.), and GIS applications (WebGIS, and etc.) do.

How can I solve this problem?

Thanks for your help.

Golden Lee

Answer 1

If you're OK with region co-ordinates being multiples of 8, the JPEG library from ijg may be able to help you load partial JPEG images.

You'd want to:

1. Get all DCT coefficients for the entire image. Here's an example of how to do this. Yes, this will involve entropy decoding of the entire image, but this is the less expensive step of JPEG decoding (IDCT is the most expensive one, and we're avoiding it).
2. Throw away the blocks that you don't need (each block consists of 8x8) coefficients. You'll have to do this by hand, but since the layout is quite simple (the blocks are in scanline order) it shouldn't be that hard.
3. Apply block inverse DCT to each of the frames. You can probably get IJG to do that for you. If you can't, then you'll have to do your own IDCT and color transform back to [0, 255] because intensities are in [-127, 128] in the world of JPEG.
4. If all goes well, you'll get your decoded JPEG image. Because of chroma subsampling, the luma and chroma channels may be of different dimensions, and you will have to compensate for this yourself by scaling.

The first two steps are pretty much covered by the links. The fourth one is quite trivial (you can get the type of chroma subsampling using the IJG interface, and scaling -- essentially upsampling -- is easily achieved by using something like OpenCV or rolling your own code). The third one is something I haven't tried yet, but it sounds like it would be possible.