Python: fetching urllib3 request headers

Question

We are injecting tracing information into request headers of all the http request calls in our API client library which is implemented based on urllib3

def _init_jaeger_tracer():
    '''Jaeger tracer initialization'''
    config = Config(
        config={
            'sampler': {
                'type': 'const',
                'param': 1,
            },
        },
        service_name="session"
        )
    return config.new_tracer()

class APIObject(rest.APIObject):
    '''Class for injecting traces into urllib3 request headers'''
    def __init__(self, configuration):
        print("RESTClientObject child class called####")
        self._tracer = None
        super().__init__(configuration)
        self._tracer = _init_jaeger_tracer()

    # pylint: disable=W0221
    def request(self, method, url, *args, **kwargs):
        lower_method = method.lower()
        with self._tracer.start_active_span('requests.{}'.format(lower_method)) as scope:
            span = scope.span
            span.set_tag(tags.SPAN_KIND, tags.SPAN_KIND_RPC_CLIENT)
            span.set_tag(tags.COMPONENT, 'request')
            span.set_tag(tags.HTTP_METHOD, lower_method)
            span.set_tag(tags.HTTP_URL, url)
            headers = kwargs.setdefault('headers', {})
            self._tracer.inject(span.context, Format.HTTP_HEADERS, headers)
    return r

After such implementation, urllib3 request headers look like this,

headers = {
    'Content-Type': 'application/json', 
    'User-Agent': 'API-Generator/1.0.0/python', 
    # **
    'uber-trace-id': '30cef3e816482516:1a4fed2c4863b2f6:0:1'
    # **
}

Now we have to obtain trace-id from the injected request header.

In python "requests" library we have "response.request.headers" attribute to return request headers

In urllib3 i could not find a way return request headers for further processing

Could any of you share some thoughts

Now from this request headers, we need to fetch the 'uber-trace-id' and its value for further building URL.

Answer 1

The documentation of urllib3.response.HTTPResponse says it's:

Backwards-compatible with http.client.HTTPResponse [...]

That's stdlib class which has getheader method described as:

Return the value of the header name, or default if there is no header matching name. If there is more than one header with the name name, return all of the values joined by ', '. If default is any iterable other than a single string, its elements are similarly returned joined by commas.

Hence in your case it's something like this.

import urllib3

http = urllib3.PoolManager()
response = http.request('GET', 'https://python.org/')
response.getheader('uber-trace-id', 'UNKNOWN')

Update

urllib3 doesn't have HTTP request object representation. An HTTP request is represented in the formal arguments of RequestMethods.request . Hence, you need to store the tracing identifier yourself in your request override after it was injected by jaeger_client. Storing it as an attribute of the response object is probably a good idea:

from jaeger_client.constants import TRACE_ID_HEADER

class APIObject(rest.APIObject):

    def request(self, *args, **kwargs) -> urllib3.response.BaseHTTPResponse:
        ...
        r._request_tracing_id = headers[TRACE_ID_HEADER]
        return r