HTTP disconnect/timeout between request and response handling

Question

Assume following scenario:

1. Client is sending HTTP POST to server
2. Request is valid and have been processed by server. Data has been inserted into database.
3. Web application is responding to client
4. Client meets timeout and does not see HTTP response.

In this case we meet situation where: - client does not know if his data was valid and been inserted properly - web server (rails 3.2 application) does not show any exception, no matter if it is behind apache proxy or not

I can't find how to handle such scenario in HTTP documentation. My question are:

a) should client expect that his data MAY be processed already? (so then try for example GET request to check if data has been submitted)

b) if not (a) - should server detect it? is there possibility to do it in rails? In such case changes can be reversed. In such case i would expect some kind of expection from rails application but there is not...

Answer 1

HTTP is a stateless protocol: Which means by definition you cannot know on the client side that the http-verb POST has succeeded or not.

There are some techniques that web applications use to overcome this HTTP 'feature'. They include.

• server side sessions
• cookies
• hidden variables within the form

However, none of these are really going to help with your issue. When I have run into these types of issues in the past they are almost always the result of the server taking too long to process the web request.

There is a really great quote to that I whisper to myself on sleepless nights:

“The web request is a scary place, you want to get in and out as quick as you can” - Rick Branson

You want to be getting into and out of your web request in 100 - 500 ms. You meet those numbers and you will have a web application that will behave well/play well with web servers.

To that end I would suggest that you investigate how long your post's are taking and figure out how to shorten those requests. If you are doing some serious processing on the server side before doing dbms inserts you should consider handing those off to some sort of tasking/queuing system.

An example of 'serious processing' could be some sort of image upload, possibly with some image processing after the upload. An example of a tasking and queuing solution would be: RabbitMQ and Celery

An example solution to your problem could be:

1. insert a portion of your data into the dbms ( or even faster some NoSQL solution )
2. hand off the expensive processing to a background task.
3. return to the user/web-client. ( even tho in the background the task is still running )
4. listen for the final response with ( polling, streaming or websockets) This step is not a trivial undertaking but the end result is well worth the effort.

Tighten up those web request and it will be a rare day that your client does not receive a response.

On that rare day that the client does not receive the data: How do you prevent multiple posts... I don't know anything about your data. However, there are some schema related things that you can do to uniquely identify your post. i.e. figure out on the server side if the data is an update or a create.

This answer covers some of the polling / streaming / websockets techniques you can use.