Python 2.7 - Redirect handler isn't passing parameters on re-direct

Question

I have a url that I am hitting on a site that can be moved, and when the endpoint is moved, I need the POST/GET parameters to be reapplied. What am I missing to ensure this handler does this?

class RedirectHandler(urllib2.HTTPRedirectHandler):


  def http_error_301(self, req, fp, code, msg, headers):
        result = urllib2.HTTPRedirectHandler.http_error_301(
            self, req, fp, code, msg, headers)
        result.status = code
        return result

    def http_error_302(self, req, fp, code, msg, headers):
        result = urllib2.HTTPRedirectHandler.http_error_302(
            self, req, fp, code, msg, headers)
        result.status = code
        return result

When I watch the traffic through fiddler, I notice that the token, which is used for authentication gets dropped.

(please note I cannot use requests for this solution, it must be standard library only)

Thank you

Answer 1

The story about HTTP 1.0 and 1.1 status codes 302, 303 and 307 is a little complicated. Basically you see the expected and documented behaviour (you may also look at this answer for longer description):

The default implementation of this method does not strictly follow RFC 2616, which says that 301 and 302 responses to POST requests must not be automatically redirected without confirmation by the user. In reality, browsers do allow automatic redirection of these responses, changing the POST to a GET, and the default implementation reproduces this behavior.

And you go the right way, but override the wrong methods. Here's the source of urllib2.HTTPRedirectHandler.redirect_request:

def redirect_request(self, req, fp, code, msg, headers, newurl):
    """Return a Request or None in response to a redirect.
    ...
    Return None if you can't but another Handler might.
    """
    m = req.get_method()
    if (code in (301, 302, 303, 307) and m in ("GET", "HEAD")
        or code in (301, 302, 303) and m == "POST"):
        # ...
        newurl = newurl.replace(' ', '%20')
        newheaders = dict((k,v) for k,v in req.headers.items()
                          if k.lower() not in ("content-length", "content-type")
                         )
        return Request(newurl,
                       headers=newheaders,
                       origin_req_host=req.get_origin_req_host(),
                       unverifiable=True)
    else:
        raise HTTPError(req.get_full_url(), code, msg, headers, fp)

Several observations here. It doesn't pass data, thus new request is GET. It filters out content-length and content-type headers, which are required for a correct POST. If fact, in my example req.headers is an empty dict, so I resorted to req.header_items() (see unredirected_hdrs). Moreover is doesn't handle POST and 307 redirect.

Here's a correct redirector handler implementation for POST and 302 redirect. Here's also complete CherryPy simulation (do pip install cherrypy before).

#!/usr/bin/env python
# -*- coding: utf-8 -*-


import urllib2
from urllib2 import HTTPRedirectHandler, Request

import cherrypy


config = {
  'global' : {
    'server.socket_host' : '127.0.0.1',
    'server.socket_port' : 8080,
    'server.thread_pool' : 8
  }
}


class RedirectHandler(HTTPRedirectHandler):

    def redirect_request(self, req, fp, code, msg, headers, newurl):
      if code == 302 and req.get_method() == 'POST':
        return Request(newurl, headers=dict(req.header_items()), data=req.data,
          origin_req_host=req.get_origin_req_host(), unverifiable=True)
      else:
        return HTTPRedirectHandler.redirect_request(self, req, fp, code, msg, 
          headers, newurl)


class App:

  @cherrypy.expose
  def index(self):
    opener = urllib2.build_opener(RedirectHandler())
    return opener.open('http://localhost:8080/redirect', data='foo=bar')

  @cherrypy.expose
  def redirect(self, **kwargs):
    print('Before redirect {0}'.format(kwargs))
    raise cherrypy.HTTPRedirect('/target', 302)
  
  @cherrypy.expose
  def target(self, **kwargs):
    return 'Target received {0} {1}'.format(cherrypy.request.method, kwargs)


if __name__ == '__main__':
  cherrypy.quickstart(App(), '/', config)