Implementing automatic rotation of JPEG images according to EXIF Orientation in ImageIO plugin

Question

Background

I've been working on an ImageIO plugin for reading JPEGs for a while. One of the features I'd like to implement, is automatic rotating of the image, according to EXIF/TIFF Orientation tag, if present.

I'm not asking how to implement the rotation itself, rather my issue is with the ImageReader API. For many users, simply calling ImageIO.read(myJPEG) is all they ever do, and I think thery're happy with having the image rotated to the correct orientation. However, if you look at the ImageReader API, it has many methods to get information and metadata about the image, and also various ways to read the pixel data.

The question(s)

Should I expose a way to read the data as is in the file, using ImageReadParam (I could do it the other way around too, but that would not allow the common case mentioned above to have the benefit of the functionality)?

I'm working on metadata support (as in IIOMetadata), should the metadata report the orientation tag as written in the file, or updated to reflect the rotated image? This is where I feel things break down... The real issue here, is what happens when someone reads the image and metadata, does some manipulations to the image and stores it again with the metadata (should the writer then do the oposite rotation, or 'magically' discard the orientation tag and update w/h)?

...or optionally, should I just give it up, and leave it to clients to read the EXIF metadata, and apply the rotation themselves if needed?

Input and suggestions from implementors as well as ImageIO users are welcome! :-)

Answer 1

Reasons why not to rotate the image

• The orientation is stored in Exif metadata, what is not part of the JPEG/JFIF standard
• Reading the Exif metadata adds additional complexity
• You can get it wrong if Exif metadata is wrong
• You have to modify the Exif metadata if you include it in the modified file.
• Rotating the image might be costly in terms of cpu load and memory
• Undefined behaviour if you read tiles

Reasons why to rotate the image

• Exif metadata is written by almost all digital cameras manufactured after 2010
• Most digital cameras do not rotate the image but store only the orientation in the metadata. Probably because additional hardware would be needed.
• Because of the last two reasons the Exif orientation can be considered a standard and a piece of information that must be respected.
• Many viewers for non-technical users like web browsers rotate the image according to Exif to provide better user experience. Consequently users will consider a view that does not rotate to be incomplete or broken.
• Other metadata information like colour conversion is also applied during reading.

I think ImageIO should rotate the image and update the Exif. That way the developer can work with ImageIO without worrying about the image file format. And the behaviour of the new JPEGReader is compatible with the old one.

Additional features would be

• to provide a method to read the image unrotated and leave the exif untouched.
• to provide a method to write the image with a specifed orientation.
• I would not make the JPEG write change its behaviour according to the Exif metadata because there can be a lot of stuff in Exif.

I think we should leave the past behind (Exif not beeing standard) and consider the orientation in Exif to be part of JPEG. But I think we still should ignore all the other Exif information when putting pixels into memory or on the screen ;-)

Answer 2

Given that every image reader in the World automagically flips the image correctly, I'm betting that virtually everybody would be fine with this happening automatically. In a sense, the image "is" the it's rotated version even if the bitmap portion of the image file says different. The whole file is the standard.

And yes, I'd be very happy if ImageIO.read() would just, um, work. It should give me an image that is as it would be displayed by every known viewer on the planet. Sure, include some APIs to get at it the raw way if somebody really wants that, but the default behavior should be the prevailing usage pattern, and the way that makes sense. Why would you ever have a need to view it in the wrong orientation? There might be odd use cases, but I bet they are very rare compared to the normal one.

Any idea when they are going to fix this in the default JDK? Writing my own code to do my own rotation to put the image back felt sick and wrong.