SqlClient hangs while executing stored procedure on SQL DB with Always Encrypted

[MForghieri]

Describe the bug

Application remains pending while executing ExecuteReader() or other commands upon an encrypted column. The issue can only be reproduced after several hours (usually we wait 24 hours but maybe less are needed) in which the application does not access the encrypted columns. This suggests a session problem.
Once the problem occurs each subsequent call remains pending, the only solution is to restart the microservice.

Adding a probe solves the problem but it is not possible to apply probes to all our microservices.

To reproduce

Below is the piece of code where the service hangs:

...
 using var command = _localContext.Database.GetDbConnection().CreateCommand();
 command.CommandType = CommandType.StoredProcedure;
 command.CommandText = name;
 command.Parameters.AddRange(TransformParameters(parameters));

 if (cancellationToken.IsCancellationRequested) return default;

 try
 {
     await _localContext.Database.OpenConnectionAsync(cancellationToken);
     using (var result = command.**ExecuteReader**())
     {
         var entities = new List<TResult>();
         while (result.Read())
         {
             entities.Add(map(result));
         }
         return entities;
     }
 }
...

We use key vault as a key store provider.
The method used to register the key store provider is as follows:

 private static void RegisterColumnEncryptionKeyStoreProviders()
        {
            var defaultAzureCredentialOptions = new DefaultAzureCredentialOptions
            {
                ExcludeEnvironmentCredential = true,
                ExcludeVisualStudioCodeCredential = true,
                ExcludeAzureCliCredential = true,
                ExcludeInteractiveBrowserCredential = true,
#if DEBUG
                ExcludeManagedIdentityCredential = true,
                ExcludeSharedTokenCacheCredential = true
#endif
            };

            DefaultAzureCredential dac = new DefaultAzureCredential(defaultAzureCredentialOptions);

            SqlColumnEncryptionAzureKeyVaultProvider sqlColumnEncryptionAzureKeyVaultProvider = new SqlColumnEncryptionAzureKeyVaultProvider(dac);
            SqlConnection.RegisterColumnEncryptionKeyStoreProviders(customProviders: new Dictionary<string, SqlColumnEncryptionKeyStoreProvider>(capacity: 1, comparer: StringComparer.OrdinalIgnoreCase)
                {
                    { SqlColumnEncryptionAzureKeyVaultProvider.ProviderName, sqlColumnEncryptionAzureKeyVaultProvider}
                });
        }

The microservice is released on azure kubernetes service.

Expected behavior

The sdk should throw an exception or execute the command but not hang

Further technical details

.NET target: .NET 8
sdk references:
Microsoft.Data.SqlClient v5.2.1
Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider v5.1.0
Microsoft.EntityFrameworkCore.SqlServer v8.0.6
Microsoft.EntityFrameworkCore v8.0.6

SQL DB references:
SQL Server version: Azure SQL Database, DC-series hardware, Always Encrypted with SGX secure enclaves

Additional context
the last log we see when the issue occur is this one:

From the logs we see that for each call that remains pending we find a call to the stored procedure ‘sp_describe_paramener_encryption’ but then the actual SP is not executed.

Together with the Microsoft support team, we have established that the issue is not related to azure services but to the sdk