🐛 Bug Report: Errors in metrics in the instrumentation of bedrock

[naghost-dev]

Which component is this bug for?

Bedrock Instrumentation

📜 Description

Currently, the instrumentation of bedrock, the responses are not being handled properly, for example the response from amazon.titan-embed-image-v1 when not returning the 'results' field returns the error sum(int(result.get("tokenCount")) for result in response_body.get("results")), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable

👟 Reproduction steps

1. Execute the model invocation: amazon.titan-embed-image-v1

👍 Expected behavior

It should correctly return the information in the trace spans and metrics

👎 Actual Behavior with Screenshots

It currently returns an exception

🤖 Python Version

3.11

📃 Provide any additional context for the Bug.

No response

👀 Have you spent some time to check if this bug has been raised before?

• I checked and didn't find similar issue

Are you willing to submit PR?

Yes I am willing to submit a PR!

[dosubot]

Hey there, @naghost-dev! 👋 I'm here to help you with bug fixes, answer your questions, and guide you on becoming a contributor. Let's work together to resolve this issue.

The error you're encountering is due to the response_body.get("results") returning None, which leads to a TypeError when trying to iterate over it. To handle this, you can modify the _set_amazon_span_attributes function to check if response_body.get("results") is None before attempting to iterate over it. Here's a possible solution:

def _set_amazon_span_attributes(span, request_body, response_body, metric_params):
    _set_span_attribute(
        span, SpanAttributes.LLM_REQUEST_TYPE, LLMRequestTypeValues.COMPLETION.value
    )
    config = request_body.get("textGenerationConfig", {})
    _set_span_attribute(span, SpanAttributes.LLM_REQUEST_TOP_P, config.get("topP"))
    _set_span_attribute(
        span, SpanAttributes.LLM_REQUEST_TEMPERATURE, config.get("temperature")
    )
    _set_span_attribute(
        span, SpanAttributes.LLM_REQUEST_MAX_TOKENS, config.get("maxTokenCount")
    )

    results = response_body.get("results")
    if results is not None:
        _record_usage_to_span(
            span,
            response_body.get("inputTextTokenCount"),
            sum(int(result.get("tokenCount")) for result in results),
            metric_params,
        )

    if should_send_prompts():
        _set_span_attribute(
            span, f"{SpanAttributes.LLM_PROMPTS}.0.user", request_body.get("inputText")
        )

        if results is not None:
            for i, result in enumerate(results):
                _set_span_attribute(
                    span,
                    f"{SpanAttributes.LLM_COMPLETIONS}.{i}.content",
                    result.get("outputText"),
                )

This modification checks if results is None before iterating over it, preventing the TypeError. You can implement this change and test it to ensure it resolves the issue.

To continue talking to Dosu, mention @dosu.

[naghost-dev]

PR#2139