Assistant API should support images in base64 if chat completion does.

[ankitmplivo]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

I tried sending base 64 image via chat completion api, and it worked. When i tried the same via assistant api for the same model, it did not work. Looking at the implementation, since it's already supported by completions api.

To Reproduce

1. Create a base64 image url (data:gdhf..) for any image.
2. Pass this base64 as url in content to chat completions API it works fine and i get a response.
3. Create an assistant.
4. Create a thread with message containing the same base64 encoded image as url inside image_url. You get an error.

BadRequestError: Error code: 400 - {'error': {'message': "Invalid 'messages[0].content[1].image_url.url'. Expected a valid URL, but got a value with an invalid format.", 'type': 'invalid_request_error', 'param': 'messages[0].content[1].image_url.url', 'code': 'invalid_value'}}

Code snippets

[{'role': 'user',
  'content': [{'type': 'text', 'text': 'What’s in this image?'},
   {'type': 'image_url',
    'image_url': {'url': 'data:image/jpeg;base64,/9j...
}
}]
}
]

OS

MacOS

Python version

Python v3.12

Library version

latest

[Lee-daeho]

I'm struggling with the same problem.
is there any progress?

[marioresl]

same problem for me i used the same format as for the completions api but it doesnt work with the base64. I think i will use the filestorage and provide the id for the assistants api. I'm really new to the OpenAI API so i dont know how to work with it properly xD

[Lee-daeho]

@marioresl
I'm new to the API especially Assistant too!
I just turned into using file.id.
It works well unless code is a little bit more messy and still have no idea why url does not works ;)

[RobertCraigie]

Thanks for reporting!

This sounds like an issue with the underlying OpenAI API and not the SDK, so I'm going to go ahead and close this issue.

Would you mind reposting at community.openai.com?

[MohamedAbdElgni]

any updates?

[charmandercha]

any updates?