.Net: New Feature: OpenAIAssistantAgent's AddChatMessageAsync for files.

[aeras3637]

---

name: OpenAIAssistantAgent's AddChatMessageAsync for files.
Please use OpenAIAssistantAgent's AddChatMessageAsync to correspond to the file.

You may want to upload a file in the middle of an Agent's conversation.
In AssistantMessageFactory.CreateOptions, only Metadata is supported and there is no way to pass FileId.
If FileReferenceContent is included in Items of ChatMessageContent, please set it to Attachments in MessageCreationOptions.

---

[crickman]

Hi @aeras3637, thank you for this suggestion. I'm curious on which type of file you are aiming to upload?

The Assistant API support both text and image content for input messages; although, Azure services / deployments are restricted to text-only.

All other file content must be either referenced from a Vector Store (for the file-search tool) or as a tool resource (for the code-interpreter tool). You may add files to a referenced vector store at any point using the OpenAIFileClient , as shown here:

https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/GettingStartedWithAgents/Step11_AssistantTool_FileSearch.cs

Additionally, files specific to a particular thread may be specified upon thread-creation via: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/Agents/OpenAI/OpenAIThreadCreationOptions.cs

(Either code-interpreter files, or a Vector Store for file-search).

From a design perspective, we chose to avoid managing , obfuscating, or otherwise interfering with how a developer might defined and update a Vector Store.

In terms of providing file-content to an assistant, we are constrained by what is possible via the API:

https://platform.openai.com/docs/api-reference/assistants

With that said, I'd very much be open to continuing to explore your proposal. Looking forward to your response.

[aeras3637]

Hi @crickman, Thank you for your response.

The files to be uploaded are in formats such as CSV and XLSX.

At the start of the chat, I specify the FileId uploaded with OpenAIFileClient to CodeInterpreterFileIds in OpenAIAssistantAgent.CreateAsync. I create an agent, create a thread with CreateThreadAsync, and execute InvokeAsync.

What I want to achieve is that, although I initially uploaded a file, there are cases where I want to upload additional files during the course of the chat conversation. However, I couldn't find a method to upload files in such cases.

[crickman]

I see, thank you. You can use the AssistantClient to modify the thread.

https://platform.openai.com/docs/api-reference/threads/modifyThread

Currently we don't directly expose patterns for thread modification. At this point in time, you can access the AssistantClient directly for this using the same OpenAIClientProvider you used to create the agent and access the OpenAIFileCient

  AssistantClient assistantClient = provider.Client.GetAssistantClient();
  await assistantClient.ModifyThreadAsync(threadId, new ThreadModificationOptions() { ToolResources = { CodeInterpreter = { FileIds = { newFileId } } } });

[aeras3637]

Thank you.

I was able to add the file with assistantClient.ModifyThreadAsync.

When adding files from the conversation flow, it feels more natural to be able to add files with AddChatMessageAsync, do you plan to support this?
The API seems to support this.
https://platform.openai.com/docs/api-reference/messages/createMessage#messages-createmessage-attachments

[crickman]

I've left this issue open for consideration. I'm not entirely sure on the full impact of integrating this as part of create message loop, but we also have a request to support modify assistant scenario...so I'll be looking at modify cases broadly.