Menu
I have to unzip and then (after processing) again zip(archive) the source files. File sizes are huge typically around 200-250 GB (unzipped, .dat format)(total 96 files). The process of unzipping takes around 2 hours and again the zipping process takes 1:30 to 2 hours which is unaffordable. Currently I am using "zcat" command for unzipping and "gzip -3" for zipping the files. Disk space is not a issue as we have 1.5 Terabyte mount in place. Will you please suggest some more efficient modes of doing this process..
Looking forward to your suggestions, Thanks - Pushkar.
783
The problem has to do with the Attachment Manager in Microsoft Windows. It flags any file that was downloaded from the internet or sent from another computer as a security risk. Unzipping software can now take a really long time to parse each file while it is unzipping.
1) Instead of right clicking on the file to extract it, left click on the file to enter the zip directly. Then look for long folder names and make them shorter. Then close the zip file and extract it again.
The ZIP file format uses lossless compression algorithms to do exactly that. It allows you to express the same information in a more efficient way by removing the redundant data from the file. This also means it is faster to send a ZIP file.
Try the silent mode -q when unzipping. This will reduce the time by a lot if there are too many files in the archive, since unzip writes the names to stdout.
man unzip:
-q perform operations quietly (-qq = even quieter). Ordinarily
unzip prints the names of the files it's extracting or testing,
the extraction methods, any file or zipfile comments that may be
stored in the archive, and possibly a summary when finished with
each archive. The -q[q] options suppress the printing of some
or all of these messages.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
