Max retries exceeded with url Tensorflow Serving Python Request

Question

I've trained my own model and I've runned docker container with this code:

docker run -t --rm -d -p 8501:8501 -v "%cd%/serving/tensorflow_serving/servables/tensorflow/testdata/optik_gpu:/models/optik_gpu" -e MODEL_NAME=optik_gpu tensorflow/serving

I'm using Windows 10 as OS.

And I'm trying to post base64 string to the container to get a prediction. Here is my code as suggested on Google Cloud:

import base64 import io import json import requests

def container_predict(image_file_path, image_key, port_number=8501):

    with io.open(image_file_path, 'rb') as image_file:
        encoded_image = base64.b64encode(image_file.read()).decode('utf-8')

    instances = {
            'instances': [
                    {'image_bytes': {'b64': str(encoded_image)},
                     'key': image_key}
            ]
    }

    url = 'http://localhost:{}/v1/models/default:predict'.format(port_number)

    response = requests.post(url, data=json.dumps(instances))
    print(response.json())

container_predict("./test.jpg", 1)

When I run the script it gives this error:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 157, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
    raise err
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 956, in send
    self.connect()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 184, in connect
    conn = self._new_conn()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001A75A6B7668>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout

(optik3) E:\Programming\Tensorflow\optik>python try.py
Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 157, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
    raise err
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request    
    conn.request(method, url, **httplib_request_kw)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 956, in send
    self.connect()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 184, in connect
    conn = self._new_conn()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001BDEB7466D8>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\retry.py", line 436, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4444): Max retries exceeded with url: /v1/models/default:predict (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001BDEB7466D8>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "try.py", line 39, in <module>
    container_predict("./test.jpg", 1, 4444)
  File "try.py", line 36, in container_predict
    response = requests.post(url, data=json.dumps(instances))
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4444): Max retries exceeded with url: /v1/models/default:predict (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001BDEB7466D8>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı'))

I don't know what is the problem.

Answer 1

I will preface this by saying I am new to all of this and am not sure how everything works.

Just had a similar problem and had a look to see what ports were open. Turns out that my docker container was on 8502 (don't know why as I thought I had specified it be on 8501 like in the tutorial here https://cloud.google.com/vision/automl/docs/containers-gcs-tutorial?hl=en_GB ). So once I changed the port to 8502 in the python code it started working.

Not sure if it will help you, but my code is now working... now I just need to figure out what is going on and why it happened!