How can ant compile-and-jar byte-identical jar files, i.e. so MD5 matches unless .java (and thus .class) changes?

Question

Summary

How can you make ant repeatedly generate byte-identical jar files from the same .class files?

Background

Our build process does the following:

1. gets web-services-definition (wsdl) files from another application's source repository
2. runs wsdl2java to generate .java file for use by web-service clients (i.e. our app)
3. compiles the java files
4. generates a .jar file from the compiler output
5. checks the 'artifact' jar file into source control

Note: We do this last step so developers have access to this jar file w/o building it themselves. We use a special 'derived' directory to distinguish source from artifacts.

The problem

We cannot get ant to generate byte-identical .jar files, even if the source files have not changed, i.e. each build generates a slightly different jar (with different MD5)

I checked the internet and found this question from some 5 years back:

If I compile some code and create a jar and related md5 file using ANT the checksum in the md5 file is different everytime even though the code hasn't changed. Any idea's why this is so how it can circumvented ? I suspect there is some timestamp information coming in somewhere.

http://www.velocityreviews.com/forums/t150783-creating-new-jar-same-code-different-md5.html

Per the responses, I've attempted the following:

1. setting the timestamp to '0' on all .class files before jarring
2. specifying a manifest file and also setting the timestamp to 0 for this manifest

[Note: this second step seems ineffective. See below]

After each build, the .jar file still has a different MD5 sum.

CSI: Jar file

I've unjarred and examined and the jars both contents and timestamps match between the "different" jars with one exception: different timestamps for META-INF/MANIFEST.MF.

Code

   <-- touch classes and manifest to set consistent timestamp across builds -->
   <touch millis="0">
    <fileset dir="${mycompany.ws.classes.dir}"/>
   </touch>
   <touch millis="0" file="mymanifest.mf"/>

   <jar destfile="${derived.lib.dir}/mycompanyws.jar"
        manifest="mymanifest.mf"
        basedir="${mycompany.ws.classes.dir}"
        includes="**/com/mycompany/**,**/org/apache/xml/**" 
    />

Other Options

We could use fancier ant programming to only check in the .jar file if the .java files have changed.

Answer 1

I have been facing a similar problem, yet slightly different. I decided to share it here as it relates to the topic of the question. In order to produce two byte-identical digitally signed JAR files in a different time one has to take the following points to consideration:

• Timestamps: **/*.class files have to have the same timestamp (java.util.zip.ZipEntry.setTime(long)). In addition, the META-INF/MANIFEST.MF file and the certificate files (*.RSA, *.DSA, and *.SF) are added to the JAR file with a "now" timestamp. So even if you decide not to compile the classes and use the ones already compiled (i.e. the ones with the original JAR's timestamp), your resulting JAR will be binary different.
• MANIFEST.MF Entries Ordering: Note that the key-value pairs in the MANIFEST.MF file are represented as a java.util.HashMap which "does not guarantee that the order will remain constant over time.". So you may run into another binary difference when signing the JAR files using JDK v5 and JDK v6 jarsigner tool as the order of the MANIFEST.MF entries may change (http://stackoverflow.com/questions/1879897/order-of-items-in-a-hashmap-differ-when-the-same-program-is-run-in-jvm5-vs-jvm6).

So basically there are two levels of the problem. Firstly, the JAR/ZIP tool that packages the files with their file-system timestamps and, thus, creates binary different JAR files for the same set of Java classes that are binary equal, but were compiled in a different time. Secondly, the JAR signer tool that modifies the META-INF/MANIFEST-MF file and appends more files to the JAR archive (certificates and class file check-sums).

The solution maybe a custom JAR signer, that sets the timestamps of all the JAR file items to a constant time and orders the MANIFEST.MF file entries (e.g. by alphabet). So far, this is, according to my knowledge, the only way to producing two byte-identical digitally signed JAR files in different time points.