Calling GetTrainedModelsAsync leads to UnexpectedTransportException

[dav-u]

Elastic.Clients.Elasticsearch version: 8.15.8 (or 8.15.1)

Elasticsearch version: 8.15.1

.NET runtime version: 8.0.10

Operating system version: Windows 10.0.19045

Description of the problem including expected versus actual behavior:
Calling

var response = await elasticsearchClient.MachineLearning.GetTrainedModelsAsync(".elser_model_2").ConfigureAwait(false);

leads to an UnexpectedTransportException.

Steps to reproduce:
Call

var response = await elasticsearchClient.MachineLearning.GetTrainedModelsAsync(".elser_model_2").ConfigureAwait(false);

Expected behavior
No exception. Instead the request is executed.

Issue #8180 is probably a similar bug.

StackTrace:

Elastic.Transport.UnexpectedTransportException: The JSON value could not be converted to System.Nullable`1[System.DateTimeOffset]. Path: $.trained_model_configs[0].create_time | LineNumber: 0 | BytePositionInLine: 451.
 ---> System.Text.Json.JsonException: The JSON value could not be converted to System.Nullable`1[System.DateTimeOffset]. Path: $.trained_model_configs[0].create_time | LineNumber: 0 | BytePositionInLine: 451.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'Number' as a string.
   at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_ExpectedString(JsonTokenType tokenType)
   at System.Text.Json.Utf8JsonReader.TryGetDateTimeOffset(DateTimeOffset& value)
   at System.Text.Json.Utf8JsonReader.GetDateTimeOffset()
   at System.Text.Json.Serialization.Converters.NullableConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseBuilder`1.ToResponseAsync[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String mimeType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultRequestPipeline`1.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, RequestData requestData, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elastic.Transport.DistributedTransport`1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List`1 seenExceptions, RequestData requestData, TResponse response, RequestPipeline pipeline)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, HttpMethod method, String path, PostData data, RequestParameters requestParameters, OpenTelemetryData openTelemetryData, CancellationToken cancellationToken)
   at Elastic.Clients.Elasticsearch.ElasticsearchClient.<>c__DisplayClass28_0`3.<<DoRequestCoreAsync>g__SendRequest|0>d.MoveNext() in /_/src/Elastic.Clients.Elasticsearch/Client/ElasticsearchClient.cs:line 158
--- End of stack trace from previous location ---

[flobernd]

Hi @dav-u, could you please send me the JSON response for this query? (e.g. by manually calling the endpoint via CURL or the Kibana Dev Console).

[flobernd]

Nevermind, I just identified the issue.