Python requests - retry for any status code but not 400

Question

I am using the following snippet(please note, only partial snippets shown):

from urllib3.util import Retry
status_forcelist = (500, 502, 504)


retry = Retry(
        total=retries,
        read=retries,
        connect=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
        method_whitelist=frozenset(['GET', 'POST'])

As of now, the retry would be done for status codes 500,502,504. However, I intend to retry for any status code but not 400. Is there a graceful way to achieve this instead of populating/hardcoding the status_forcelist for all status codes ?

Answer 1

You can take the codes from request module - which saves you writing them out. In case any new ones get introduced you would have them as well:

import requests
from urllib3.util import Retry

status_forcelist = tuple( x for x in requests.status_codes._codes if x != 400)

print(status_forcelist)

Output:

(100, 101, 102, 103, 122, 200, 201, 202, 203, 204, 205, 206, 207, 208, 226, 300, 301, 302, 
 303, 304, 305, 306, 307, 308, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 
 413, 414, 415, 416, 417, 418, 421, 422, 423, 424, 425, 426, 428, 429, 431, 444, 449, 450, 
 451, 499, 500, 501, 502, 503, 504, 505, 506, 507, 509, 510, 511)

Caveat: wikipedia list "self created" / unofficial statuscodes as well as proprietary ones around 9xx which are not covered by this list though. @zwer solution would cover those as well.

Answer 2

status_forcelist, unfortunately, applies as a whitelist only (given the use, it can be called a blacklist, tho) so there are no built-in ways to do it gracefully. However, you can override (or monkey-patch if that's your thing) urllib3.util.retry.Retry.is_retry() to treat it as a blacklist, something like:

import urllib3.util.retry

class BlacklistRetry(urllib3.util.retry.Retry):

    def is_retry(self, method, status_code, has_retry_after=False):
        if not self._is_method_retryable(method):
            return False

        if self.status_forcelist and status_code not in self.status_forcelist:
            # threat as a blacklist --------------^
            return True

        return (self.total and self.respect_retry_after_header and
                has_retry_after and (status_code in self.RETRY_AFTER_STATUS_CODES))

Then use it instead of the original Retry and just add 400 to its status_forcelist.