Is a HTTP Get request of size 269KB allowed?

Question

I am debugging a test case. I use Python's OptionParser (from optparse) to do some testing and one of the options is a HTTP request.

The input in this specific case for the http request was 269KB in size. So my python program fails with "Argument list too long" (I verified that there was no other arguments passed, just the request and one more argument as expected by the option parser. When I throw away some of the request and reduce its size, things work fine. So I have a strong reason to believe the size of the request is causing my problems here.)

Is it possible for a HTTP request to be that big ? If so how do I fix the OptionParser to handle this input?

Answer 1

Is it possible for a HTTP request to be that big ?

Yes it's possible but it's not recommended and you could have compatibility issues depending on your web server configuration. If you need to pass large amounts of data you shouldn't use GET.

If so how do I fix the OptionParser to handle this input?

It appears that OptionParser has set its own limit well above what is considered a practical implementation. I think the only way to 'fix' this is to get the Python source code and modify it to meet your requirements. Alternatively write your own parser.

UPDATE: I possibly mis-interpreted the question and the comment from Padraic below may well be correct. If you have hit an OS limit for command line argument size then it is not an OptionParser issue but something much more fundamental to your system design that means you may have to rethink your solution. This also possibly explains why you are attempting to use GET in your application (so you can pass it on the command line?)