Attach custom thumbnail for a video (not extract) - IOS, AVFoundation

Question

I generate video by using AVFoundation. After that I write video to the Photos library by using Photos Framework (and get instance of PHAsset after that).

I want to set custom thumbnail for added PHAsset (last frame from the video), but can't find a solution. How I can attach custom thumbnail for added video? I want to see my custom thumbnail in Photos application, when I'm opening in.

Also, I know how to get some image from the video by using AVAssetImageGenerator, but I want to see my thumbnail in Photo application.

Answer 1

To add a custom thumbnail to your MP4:

METHOD 1: using AVMutableMetadataItem...

See if the code in this Question helps you.

AVMutableMetadataItem *item = [[AVMutableMetadataItem alloc] init];
item.keySpace = AVMetadataKeySpaceCommon;
item.key = AVMetadataCommonKeyArtwork;
item.value = UIImageJPEGRepresentation([[UIImage alloc] initWithContentsOfFile:[[NSSearchPathForDirectoriesInDomains (NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0] stringByAppendingPathComponent:@".image"]]);

[newMetadataArray addObject:item];
assetWrtr.metadata = newMetadataArray;

[assetWrtr startWriting];
[assetWrtr startSessionAtSourceTime:kCMTimeZero];

See also this Apple forum Post. Read the whole thread for context of code & usage.

METHOD 2: manually writing bytes...

Short version :

Find the following atoms (tags)... moov goes to udta goes to meta then ilst and here, simply add a covr atom (24 bytes) followed by the bytes of your jpeg image (bytes FF D8 up to FF D9). Update the size entries (32-bit integers) for moov, udta, ilst and covr to reflect the newly added bytes). That's it. Finished.

Expanded version :

( note / recommended ):
• Ideally check a test MP4 in a hex editor (d/load a free one) to follow the bytes as described below.
• Read : Apple's Quicktime Format specs and Cimmaron System's MP4 atoms guide.

1) Find moov bytes...

(when viewed in a hex editor) Your MP4 bytes should look something like this.

00 00 00 20 66 74 79 70 69 73 6F 6D 00 00 02 00     ... ftypisom....
69 73 6F 6D 69 73 6F 32 61 76 63 31 6D 70 34 31     isomiso2avc1mp41
00 00 00 08 66 72 65 65 00 00 99 70 6D 64 61 74     ....free..™pmdat

These are the opening bytes and the important part is that is has the bytes 6D 64 61 74 meaning mdat (as ASCII characters, on right side of shown bytes view). I don't use iOS so I hope it makes mdat first & then moov is placed towards end of file, in such a case it's easy to add extra bytes without corrupting the file. If you see mdat within first 64 bytes of your file then you can proceed with my advice below.

edit: (if not already like this by default) It seems you can place mdat atom at front of file (first 64 bytes) if your exportSession settings have :

exportSession.shouldOptimizeForNetworkUse = NO; 

To find moov, read the previous 4 bytes (as one integer) before the ASCII letters "mdat". In above example, this is a 4-byte integer of 00 00 99 70 ( = 39280 bytes). This means skip a total of 39280 + 8 bytes starting from 6D.. 64.. 61 and so on. Once skipped, the next 4 bytes should be 6D 6F 6F 76 ("moov" in ASCII). Note this offset/position as moov beginning.

2) Add covr bytes...

From moov beginning, read the following bytes searching for :

• find entry udta as bytes 75 64 74 61. Note for later reference : this position - 4 as starting of udta size pos.
• find entry meta as bytes 6D 65 74 61.
• find entry ilst as bytes 69 6C 73 74. Note for later reference : this position - 4 as starting of ilst size pos.

note: If any of the above entries is not found, you must create those bytes. Check page 14 onwards of this atoms guide to know which bytes (values) are needed for those above atoms.

• at the ending of ilst add four zero bytes 00 00 00 00 (later this will be updated as total covr size). For reference, note these 4 bytes' position as covr size pos.
• add entry covr by writing bytes/integer as 63 6F 76 72.
• add bytes 00 00 ED EA 64 61 74 61 00 00 00 0D 00 00 00 00 then it's ready for JPEG image bytes.

3) Add JPEG bytes...

Paste the bytes of JPEG image. These bytes begin with FF D8 and end with FF D9. Note the total amount of these bytes as jpeg size.

4) Update sizes...

• for covr : go to the starting of covr size pos, replace the four 00 00 00 00 bytes with hex result of jpeg size + 20 calculation.
• for ilst : go to the starting of ilst size pos bytes, replace those four bytes with hex result of current ilst size + covr size + 4 calculation..
• for udta : go to the starting of udta size pos bytes, replace those four bytes with hex result of current udta size + covr size + 4 calculation..

conclusion

Test MP4 file by enabling some thumbnail view in your program/tool. You should see the jpg now used as icon for the edited mp4 file.

PS: I don't code for iOS (no Swift or Objective-C knowledge) so I can't show you an example code, only advice on creating the bytes. This task can even be done manually using a hex editor. The main thing for you as an iOS coder is to be able to write bytes to an existing file and re-save as new filename (or do overwrites when code is perfected).

Answer 2

If you are writing a part of the app which will browse the photo library and display the thumbnails, then you should be able to do this as follows:

1. get the PHAsset
2. from the PHAsset request an AVasset using requestAVAssetForVideo
3. generate a thumbnail at the time you require using AVAssetImageGenerator and the copyCGImageAtTime method

One thing to be wary of is that timing and selecting a specific frame in videos is a tricky science/art - getting exactly the frame you want from a time might prove difficult.

Also, the video will have to be decoded at least partially to get the image - the above approach is meant to leverage the frameworks to do this efficiently, but even efficient video decoding is time and power hungry.

If you are saying you want the videos to appear in the standard 'Photos' app on iOS with a custom thumbnail you assign to them, then I'm not sure how you can do that.