How to speed up HTTP Response receiving using jQuery Ajax in large data posting?

Question

Issue: python webapp2 + jQuery Ajax perfroms extremely bad in receiving large text data response (takes more than 10 minutes in a 1.7MB payload roundtrip)

Question: what's the reason? how to improve it? could I use any well-proved techniques to divide the big text trunk into small payloads to avoid the 'browser hung' issue?

Background: I've been learning python web programming with webapp2 + Google App Engine. I'm trying to build a what-you-input-is-what-you-see editing zone with jQuery Ajax. It's very similar to stackoverflow post editor: wmd-input vs wmd-preview which offers live preview function. (It keeps indicating 'draft saved' to short text. Another example is Google Docs live editing function)

My example works like this: A textchange jQuery plugin fires Ajax posting triggered by each input textarea change ---> Python backends receive the text and add some messages upon it ---> Send back the text+messages ---> jQuery use the server response to update the preview textarea (Well, sending back the full content of received text is just for test purpose.)

My frontend codes:

<script type="text/javascript">
function simpleajax() {
        $.ajax({
            type: 'POST'
            ,url: '/simpleajax'
            ,dataType: 'json'
            ,data:{'esid':'#ajaxin','msgin':$('#ajaxin').val()}
            ,cache:false
            ,async:true
            ,success:function (resp){$('#ajaxout').text(resp.msgout);}
            ,error:function (jqXHR, textStatus, errorThrown){
                {$('#ajaxout').text("Ajax Error:"+textStatus+","+errorThrown)}}
        });
    }

$(document).ready(function(){
$('#ajaxin').bind('textchange', function () {
    $('#ajaxstatus').html('<strong color="blue">Typing ...</strong>');
    simpleajax();
    });
});
</script>

My backend codes:

class simpleajax(BaseReqHandler):
    def get(self):
        content={'pagealert':'simpleAjax get response'}
        self.render_to_response('simpleAjax.html',**content)

    def post(self):
        esid=self.POST['esid']
        msgin=self.POST['msgin']
        msgout="Server noticed element " + esid + " value changed" + " and saved following message: " + msgin
        content={"msgout":msgout}
        self.writeout(content)

Test case and symptoms: Local server + plain text payloads

copy and paste plain text smaller than 500KB into input area: works like a charm. however, a 1.7MB text makes browser busy in >10minutes which appears totally no responding.

Comparison: I paste the same text to stackoverflow post editor and the preview appears instantly! I noticed no draft-saved tips this time. And there're some javascript codes judging the text length here. Well. There is a chance no server communications involved. But it's a workaround not a solution to my issue.(Google Docs autosave function must utilizes some kind of techniques to solve the issue!)

Firebug xhr monitor results:

#Request Headers:
Host: localhost:8080
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
X-Requested-With: XMLHttpRequest
Content-Length: 2075974
Referer: http://localhost:8080/ajax
Cookie: __utma=111872281.1883490050.1319630129.1319630129.1319637523.2; __utmz=111872281.1319630129.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
Pragma: no-cache
Cache-Control: no-cache

#Response Headers:
Server: Development/1.0
Date: Fri, 04 Nov 2011 03:29:05 GMT
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Content-Length: 1790407

#Firebug Timeline:
TimePoints       TimeElapsed      Actions
0                 1ms           DNS Lookup
+1ms              1ms           Connecting
+2ms              82ms          Sending
+84ms            1.03s          Waiting
+1.11s           14m22s         Receiving
+14m23.11s                       Done

Interesting things:

1. jQuery Ajax sends 2MB instead of 1.7MB pure payload to server. What a big overhead! It could be due to Content-Type: application/x-www-form-urlencoded ?
2. Server takes 1.03s to respond and jQuery takes 14 minutes to receive the response!!!

What's going on behind this? Any help are appreciated! I'd like to let server 'push' a lot of things to client after an Ajax request, but this issue makes it impossible.

Answer 1

Consider using a combination of HTML5 WebSockets & Worker API to asynchronously send/receive data from the server without affecting the UI thread performance.

http://dev.w3.org/html5/websockets/

http://net.tutsplus.com/tutorials/javascript-ajax/start-using-html5-websockets-today/ (the tutorial assumes PHP to be the server side technology)

http://www.whatwg.org/specs/web-apps/current-work/multipage/workers.html

Another Option

a) On mousedown & keyup

- record the cursor position in the text-box - store it in C1.

b) On textchange

> record the cursor position - store it in C2.

> send only the content between C1 and C2 to your server. Also send the values C1 and C2 to the server, i.e your AJAX payload will look something like: 

{ c1: C1, c2: C2, data: <text in the text-box from C1 to C2> }

You'll need to see if c1 > c2, and get the substring appropriately and vice versa

This way, only "changes" are sent to server each time - not the full text. But if you copy and paste 5MB of content, this will not give any improvement. But for single character changes (like typing, pasting small segments etc..) this should work rather well.