Menu
I want to download a huge file from an ftp server in chunks of 50-100MB each. At each point, I want to be able to set the "starting" point and the length of the chunk I want. I won't have the "previous" chunks saved locally (i.e. I can't ask the program to "resume" the download).
What is the best way of going about that? I use wget mostly, but would something else be better?
I'm really interested in a pre-built/in-build function rather than using a library for this purpose... Since wget/ftp (also, I think) allow resumption of downloads, I don't see if that would be problem... (I can't figure out from all the options though!)
I don't want to keep the entire huge file at my end, just process it in chunks... fyi all - I'm having a look at continue FTP download afther reconnect which seems interesting..
991
there is a "--continue" option available for wegt, to continue getting a partially downloade file. This is useful when you want to finish a download started by a previous instance of wget. Make sure you run the wget command with the --continue option in the same directory where the first download started.
To resume the partially downloaded file, go to the location where the partially downloaded file exists, and use -c or --continue option with wget command like below. Now, Wget started to download the file from where it was left in the previous attempt.
Press Ctrl + J or click the Options dropdown menu and select Downloads to open the download manager. In the list of downloads, find the failed item and click Resume.
To resume file transfer, a request must be resubmitted to the adapter. Set the ResumeFailedTransfer property to True in the wrapper object, for the adapter to resume the file transfer. The adapter, upon reestablishing the connection to the FTP server, resumes the transfer of the file being created on the FTP server.
Use wget with:
Extracted from man pages:
-c / --continue
Continue getting a partially-downloaded file. This is useful when you want to finish up a download started by a previous instance of Wget, or by another program. For instance:
wget -c ftp://sunsite.doc.ic.ac.uk/ls-lR.Z
If there is a file named ls-lR.Z in the current directory, Wget will assume that it is the first portion of the remote file, and will ask the server to continue the retrieval from an offset equal to the length of the local file.
160
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
