Assistant API should support images in base64 if chat completion does.

[ankitmplivo]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

I tried sending base 64 image via chat completion api, and it worked. When i tried the same via assistant api for the same model, it did not work. Looking at the implementation, since it's already supported by completions api.

To Reproduce

1. Create a base64 image url (data:gdhf..) for any image.
2. Pass this base64 as url in content to chat completions API it works fine and i get a response.
3. Create an assistant.
4. Create a thread with message containing the same base64 encoded image as url inside image_url. You get an error.

BadRequestError: Error code: 400 - {'error': {'message': "Invalid 'messages[0].content[1].image_url.url'. Expected a valid URL, but got a value with an invalid format.", 'type': 'invalid_request_error', 'param': 'messages[0].content[1].image_url.url', 'code': 'invalid_value'}}

Code snippets

[{'role': 'user',
  'content': [{'type': 'text', 'text': 'What’s in this image?'},
   {'type': 'image_url',
    'image_url': {'url': 'data:image/jpeg;base64,/9j...
}
}]
}
]

OS

MacOS

Python version

Python v3.12

Library version

latest