.Net: Bug: Qdrant CreateCollectionIfNotExistsAsync fails for the first time

[panicoenlaxbox]

Describe the bug
An error occurs when creating a Qdrant collection for the first time. However, the collection is created. A subsequent run works fine.
The problem is in await collection.CreateCollectionIfNotExistsAsync();

System.Collections.Generic.KeyNotFoundException
  HResult=0x80131577
  Message=The given key 'System.String[]' was not present in the dictionary.
  Source=System.Private.CoreLib
  StackTrace:
   at System.ThrowHelper.ThrowKeyNotFoundException[T](T key)
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Microsoft.SemanticKernel.Connectors.Qdrant.QdrantVectorStoreRecordCollection`1.<CreateCollectionAsync>d__15.MoveNext()
   at Microsoft.SemanticKernel.Connectors.Qdrant.QdrantVectorStoreRecordCollection`1.<CreateCollectionIfNotExistsAsync>d__16.MoveNext()
   at Program.<<Main>$>d__0.MoveNext() in ...

To Reproduce

#pragma warning disable SKEXP0010
#pragma warning disable SKEXP0001
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.VectorData;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.Qdrant;
using Microsoft.SemanticKernel.Embeddings;
using Qdrant.Client;

var configuration = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json", optional: true)
    .Build();

var services = new ServiceCollection();
services
    .AddTransient<IConfiguration>(_ => configuration)
    .AddLogging(configure => configure
        .AddConfiguration(configuration.GetSection("Logging"))
        .AddConsole());


var endpoint = "YOUR_ENDPOINT";
var apiKey = "YOUR_API_KEY";

services.AddAzureOpenAITextEmbeddingGeneration("text-embedding-ada-002", endpoint, apiKey);

services.AddTransient((serviceProvider) =>
{
    return new Kernel(serviceProvider);
});

using var scope = services.BuildServiceProvider().CreateScope();

var kernel = scope.ServiceProvider.GetRequiredService<Kernel>();

var vectorStore = new QdrantVectorStore(new QdrantClient("localhost"));
var collection = vectorStore.GetCollection<ulong, Customer>("skcustomers");
await collection.CreateCollectionIfNotExistsAsync();  // Here it's the error

async Task<ReadOnlyMemory<float>> GenerateEmbeddingAsync(string textToVectorize)
{
    var textEmbeddingGenerationService = scope.ServiceProvider.GetRequiredService<ITextEmbeddingGenerationService>();
    ReadOnlyMemory<float> embedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(textToVectorize);
    return embedding;
}

const ulong customerId = 1;
const string description = "It's an amazing company that helps you build new, quality applications with an agile approach.";

await collection.UpsertAsync(new Customer
{
    Id = customerId,
    Name = "BitAttack",
    Email = "[email protected]",
    Description = description,
    Tags = ["development", "software", "agile"],
    DescriptionEmbedding = await GenerateEmbeddingAsync(description),    
});

Customer? retrievedCustomer = await collection.GetAsync(customerId);
Console.WriteLine(retrievedCustomer);

public class Customer
{
    [VectorStoreRecordKey]
    public ulong Id { get; set; }

    [VectorStoreRecordData(IsFilterable = true)]
    public string Name { get; set; }

    [VectorStoreRecordData]
    public string Email { get; set; }

    [VectorStoreRecordData(IsFullTextSearchable = true)]
    public string Description { get; set; }

    [VectorStoreRecordData(IsFilterable = true)]
    public string[] Tags { get; set; }

    [VectorStoreRecordVector(Dimensions: 1536, DistanceFunction.CosineSimilarity, IndexKind.Hnsw)]
    public ReadOnlyMemory<float>? DescriptionEmbedding { get; set; }

    public override string ToString() => $"Id: {Id}, Name: {Name}, Email: {Email}, Description: {Description}, Tags: {string.Join(", ", Tags)}";
}

Expected behavior
Don't fail the first time

Screenshots
If applicable, add screenshots to help explain your problem.

Platform
Relevant packages used (I think):

	<PackageReference Include="Microsoft.Extensions.VectorData.Abstractions" Version="9.0.0-preview.1.24523.1" />
	<PackageReference Include="Microsoft.SemanticKernel" Version="1.30.0" />
	<PackageReference Include="Microsoft.SemanticKernel.Connectors.Qdrant" Version="1.30.0-preview" />

Additional context
For now, I can go ahead with this:

try
{
    await collection.CreateCollectionIfNotExistsAsync();
}
catch (KeyNotFoundException)
{
}

[panicoenlaxbox]

Hi,

In addition to that error, when I want to search it also fails.

Unhandled exception. Microsoft.Extensions.VectorData.VectorStoreOperationException: Call to vector store failed.
 ---> Grpc.Core.RpcException: Status(StatusCode="Unimplemented", Detail="")
   at Qdrant.Client.QdrantClient.QueryAsync(String collectionName, Query query, IReadOnlyList`1 prefetch, String usingVector, Filter filter, Nullable`1 scoreThreshold, SearchParams searchParams, UInt64 limit, UInt64 offset, WithPayloadSelector payloadSelector, WithVectorsSelector vectorsSelector, ReadConsistency readConsistency, ShardKeySelector shardKeySelector, LookupLocation lookupFrom, Nullable`1 timeout, CancellationToken cancellationToken)
   at Microsoft.SemanticKernel.Connectors.Qdrant.QdrantVectorStoreRecordCollection`1.RunOperationAsync[T](String operationName, Func`1 operation)
   --- End of inner exception stack trace ---
   at Microsoft.SemanticKernel.Connectors.Qdrant.QdrantVectorStoreRecordCollection`1.RunOperationAsync[T](String operationName, Func`1 operation)
   at Microsoft.SemanticKernel.Connectors.Qdrant.QdrantVectorStoreRecordCollection`1.VectorizedSearchAsync[TVector](TVector vector, VectorSearchOptions options, CancellationToken cancellationToken)
   at Program.<Main>$(String[] args) in C:\Users\***\source\repos\SemanticKernel\MyKernel\MyKernel\Program.cs:line 77
   at Program.<Main>(String[] args)

On the other hand, the same code works perfectly with Redis

I have chosen Qdrant and Redis, because they seem to be the most supported store connectors https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/

[westey-m]

@panicoenlaxbox, thanks for logging these issues and the detailed repro steps. Much appreciated. The first is definitely a miss, and I've submitted a PR to fix it.

With regards to the error you are seeing on search. What version / type of Qdrant are you using? E.g. is it docker running locally or cloud and what version are you using? From the error you are seeing it sounds like the call we are making isn't available in the service, which points at an api mismatch. Our tests are passing against the latest docker image of Qdrant.

[panicoenlaxbox]

Hi @westey-m,
I am running Qdrant in local with Docker. I had downloaded an old image:

sha256:4b3bd72db8af29db25435a965d01f033adfa796f5cbdfecdc5d5da53f67936f3
Qdrant v1.8.4

I have pulled the newest version and, the vector search works fine! :)

sha256:63343dd5a0745ba818ad3abc35f3d4aadce5b3bcdd32178e6ec6a8cf1a5f8f19
v1.12.4

Thank you so much for your support.

[westey-m]

@panicoenlaxbox, we will release the bug fix with our next release (hopefully this week). In the mean time, the bug is in the index creation for Tags, so if you remove IsFilterable from Tags, it'll work around the issue for now.

    [VectorStoreRecordData]
    public string[] Tags { get; set; }