JPG+Zip File Combination Problem with Zip Format

Question

Hopefully you've heard of the neat hack that lets you combine a JPG and a Zip file into a single file and it's a valid (or at least readable) file for both formats. Well, I realized that since JPG lets arbitrary stuff at the end, and ZIP at the beginning, you could stick one more format in there - in the middle. For the purposes of this question, assume the middle data is arbitrary binary data guarenteed not to conflict with the JPG or ZIP formats (meaning it doesn't contain the magic zip header 0x04034b50). Illustration:

0xFFD8 <- start jpg data end -> 0xFFD9 ... ARBITRARY BINARY DATA ... 0x04034b50 <- start zip file ... EOF

I am catting like this:

cat "mss_1600.jpg" filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb "null.bytes" "randomzipfile.zip" > temp.zip

This produces a 6,318 KB file. It does not open in 7-Zip. However, when I cat one less 'double' (so instead of 13 filea and b's, 12):

cat "mss_1600.jpg" filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb filea fileb "null.bytes" "randomzipfile.zip" > temp.zip

It produces a 5,996 KB file that does open in 7-Zip.

So I know my arbitrary binary data doesn't have the magic Zip File Header to screw it up. I have reference files of the working jpg+data+zip and the non-working jpg+data+zip (save-as cause the browser thinks they're images, and add the zip extensions yourself).

I want to know why it fails with 13 combinations and doesn't with 12. For bonus points, I need to get around this somehow.

Answer 1

I downloaded the source for 7-Zip and figured out what is causing this to happen.

In CPP/7zip/UI/Common/OpenArchive.cpp, you'll see the following:

// Static-SFX (for Linux) can be big.
const UInt64 kMaxCheckStartPosition = 1 << 22;

That means that only the first 4194304 bytes of the file will be searched for the header. If it isn't found there, 7-Zip considers it an invalid file.

You can double that limit by changing 1 << 22 to 1 << 23. I tested that change by rebuilding 7-Zip and it works.

EDIT: To get around this issue, you can download the source, make the above change, and build it. I built it using VS 2008. Open the VS command prompt, navigate to extracted-source-location\CPP\7zip\Bundles and type 'nmake'. Then in the Alone directory run '7za t nonworking.jpg' and you should see 'Everything is Ok'.