Best practice for loading a large text file using jQuery get()

Question

Currently I'm storing the results of the get in a string since the file I am opening is plain text file 3MB to 20MB in size.

Then I parse this string and modify it so that the end result can be outputted in html format.

I'm just looking for a sanity check to see if loading in this manner is the best way?

Also, is there a way to load a chunk of the target text file, parse the chunk, request another chunk, etc. Kind of like a music player buffering a song while playing the song.

Thank you

Answer 1

is there a way to load a chunk of the target text file, parse the chunk, request another chunk, etc.

To retrieve a portion (or chunk) of a resource, use the HTTP Range header. Most web servers honor this header correctly.

example HTTP Request:

GET /resource/url/path HTTP/1.1\r\n
User-agent: whatever\r\n
Host: www.example.com\r\n
Range: bytes=0-3200\r\n\r\n

Update:

With jQuery, v1.5 and later, you can tell jQuery to send additional HTTP headers for an individual ajax call, like this:

    $.ajax({type: "GET",
            url: url,
            headers : { "Range" : 'bytes=0-3200' },
            ....

If you don't want to modify each $.ajax() call, Stofke says you can use a "beforeSend" fn. Something like this:

  $.ajaxSetup({ beforeSend : function(xhr) {
        xhr.setRequestHeader("Range", "bytes=0-3200" );
  }});      

Using $.ajaxSetup(), you'd need to modify that range header for each additional call.

I don't know of a way to tell jQuery to load a resource chunkwise, automatically. I guess you'd have to do that yourself with setTimeout(), or something, in the success() of the ajax call. Each time you get a response, call setTimeout() and make the next ajax call. Your browser-resident Javascript will need to keep track of which portions of the file you have retrieved already, and which part (byte index) you want to get next.

Another way to do it would be to make the ajax calls only when your app is ready to do so. Rather than waiting for success() from the ajax call, just make an ajax call after you've finished processing the results of the first range retrieval.

Also: to support the arithmetic purposes, before doing the first GET, you can use the HEAD call to learn the length of the resource, which will tell you the max index in the range you can use.