Skip to content

Conversation

@roji
Copy link
Member

@roji roji commented Oct 30, 2025

  • Added new KeyTypeTests to cover provider key support, alongside the existing DataTypeTests and EmbeddingTypeTests; moved all three into TypeTests folder.
  • As discussed offline, this adds Guid-as-string key support for all providers. From research this seems to be very standard practice across a wide range of databases: JSON-based databases generally encode various types (e.g. DateTime) as string, and Guids are standard there. Sqlite is similar, with Microsoft.Data.Sqlite supporting it natively. This notably allows upper layers to assume that they can always use Guid, which helps MEDI (see this issue).
  • Per-provider notes
    • Azure AI Search: string only supported, added Guid-as-string as an option (docs encouraging mapping Guid to strings)
    • MongoDB/CosmosMongoDB: Added int and long support (Guid was already supported).
    • CosmosNoSql: Added Guid-as-string (standard practice)
    • Pinecone: Added Guid-as-string
    • Redis: Added Guid-as-string
    • SQL Server: removed DateTime and byte[] as key types (as above) and added Guid.
    • SQLite: The Microsoft.Data.Sqlite (the low-level ADO.NET provider we're built on top of) supports GUIDs as TEXT; so it's really not us deciding to support - it's the lower-level driver. In addition, we support mapping boolean to INTEGER (1/0), which seems similar. Added support as data property as well, not just key.

Closes #11784
Closes #12182

@roji roji requested a review from a team as a code owner October 30, 2025 11:26
@markwallace-microsoft markwallace-microsoft added the .NET Issue or Pull requests regarding .NET code label Oct 30, 2025
@github-actions github-actions bot changed the title [MEVD] Go over all key types, add support for Guid .Net: [MEVD] Go over all key types, add support for Guid Oct 30, 2025
@roji roji enabled auto-merge October 30, 2025 14:07
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but please take a look at my comments as some changes seem to be from a different PR. Thank you @roji for implementing this feature for the Data Ingestion project!

@roji roji added this pull request to the merge queue Oct 31, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 31, 2025
@roji roji added this pull request to the merge queue Oct 31, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 31, 2025
@roji roji added this pull request to the merge queue Oct 31, 2025
Merged via the queue into microsoft:main with commit b697358 Oct 31, 2025
18 checks passed
@roji roji deleted the KeyTypes branch October 31, 2025 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Issue or Pull requests regarding .NET code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.Net: [MEVD] Support Guid across MEVD providers (as string where needed) .Net: [MEVD] Go over the supported keys across the providers

4 participants