Where to log request and response headers of generations? (litellm)

[Manouchehri]

For generation events, it'd be nice if Langfuse could display HTTP request and response headers in a pretty way.

I know there's metadata and tags already, but I feel like for diagnostics/troubleshooting, having the actual headers displayed separately from the metadata section is easier to parse as a human.

[Manouchehri]

cc @krrishdholakia and @ishaan-jaff, would be epic to have this supported by LiteLLM if it gets added to Langfuse. =)

Related to BerriAI/litellm#1724.

[marcklingen]

For now I agree to add it to metadata of a generation: metadata["headers"]:{}

If others are interested, I would be open to adding more native support. Langfuse is highly customizable and flexible, and we want to maintain this. We only want to suggesting and standardize on "best practices" once it is clear that they are effective for most users.

[Manouchehri]

How do you think response vs request headers should be labeled in that?

I was thinking maybe like this:

{
  "request": {
    "headers": {
      "host": "example.com"
    },
    "url": "https://api.openai.com/v1/chat/completions"
  },
  "response": {
    "headers": {
      "age": "1337"
    },
    "status": 200
  }
}

This way it'd be easy to associate more than just headers with the response and request in the future.

[marcklingen]

I'd do it exactly like this 👍