Large File Download

Question

Internet Explorer has a file download limit of 4GB (2 GB on IE6). Firefox does not have this problem (haven't tested safari yet) (More info here: http://support.microsoft.com/kb/298618)

I am working on a site that will allow the user to download very large files (up to and exceeding 100GB)

What is the best way to do this without using FTP. The end user must be able to download the file from there browser using HTTP. I don't think Flash or Silverlight can save files to the client so as far as I know they won't cut it.

I'm guessing we will need an ActiveX or Java applet to pull this off. Something like the download manager that MSDN uses.

Does anyone know of a commercial (or free) component that will do that? We do not want the user to have to install a "browser wide" download manager (like GetRight), we want it to only work with downloading on our site.

Update: Here is some additional info to help clarify what I'm trying to do. Most of the files above the 4GB limit would be large HD video files (its for a video editing company). These will be downloaded by users across the internet, this isn't going to be people on a local network. We want the files to be available via HTTP (some users are going to be behind firewalls that aren't going to allow FTP, Bittorrent, etc.). The will be a library of files the end user could download, so we aren't talking about a one time large download. The will be download different large files on a semi-regular basis.

So far Vault that @Edmund-Tay suggested is the closest solution. The only problem is that it doesn't work for files larger than 4GB (it instantly fails before starting the download, they are probably using a 32bit integer somewhere which is exceeded/overflown by the content length of the file).

The best solution would be a java applet or ActiveX component, since the problem only exist in IE, that would work like the article @spoulson linked to. However, so far I haven't had any luck finding a solution that does anything like that (multipart downloads, resume, etc.).

It looks like we might have to write our own. Another option would be to write a .Net application (maybe ClickOnce) that is associated with an extension or mime type. Then the user would actually be downloading a small file from the web server that opens in the exe/ClickOnce app that tells the application what file to download. That is how the MSDN downloader works. The end user would then only have to download/install an EXE once. That would be better than downloading an exe every time they wanted to download a large file.

Answer 1

@levand:

My actual preference, as a user, in these situations is to download a lightweight .exe file that downloads the file for you.

That's a dealbreaker for many, many sites. Users either are or should be extremely reluctant to download .exe files from websites and run them willy-nilly. Even if they're not always that cautious, incautious behaviour is not something we should encourage as responsible developers.

If you're working on something along the lines of a company intranet, a .exe is potentially an okay solution, but for the public web? No way.

@TonyB:

What is the best way to do this without using FTP.

I'm sorry, but I have to ask why the requirement. Your question reads to me along the lines of "what's the best way to cook a steak without any meat or heat source?" FTP was designed for this sort of thing.

Answer 2

bittorrent?

There have been a few web-based versions already (bitlet, w3btorrent), and Azureus was built using java, so it's definitely possible.

Edit: @TonyB is it limited to port 80?