Invalid content type. image_url is only supported by certain models

Question

That's my Python request:

def analyze_screenshot_base64(encoded_image):
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {openai.api_key}"
    }

    payload = {
        "model": "gpt-4o-mini",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Develop a trading setup (LONG or SHORT) with a CRV of at least 5 based on this chart. Include entry price, stop loss, and take profit levels."
                    },
                    {
                        "type": "image_url",
                        "image": {
                            "url": f"data:image/png;base64,{encoded_image}"
                        }
                    }
                ]
            }
        ],
        "max_tokens": 300
    }

    response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload)
    return response.json()

But the API returns me:

{'error': {'message': 'Invalid content type. image_url is only supported by certain models.', 'type': 'invalid_request_error', 'param': 'messages.[0].content.[1].type', 'code': None}}

What could be the problem?

Answer 1

Although the error message seems to suggest that you might be using the wrong model, this is not the case.

When specifying the image content:

{
    "type": "image_url",
    "image": {
        "url": f"data:image/png;base64,{encoded_image}"
    }
}

you need to use image_url rather than image.

So this becomes:

{
    "type": "image_url",
    "image_url": {
        "url": f"data:image/png;base64,{encoded_image}"
    }
}

I have checked this by running part your code, and I get the same error. If I change it to image_url it works perfectly.