Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAI: Attach custom attributes to llm_events when prefixed with llm. #2437

Closed
hannahramadan opened this issue Feb 13, 2024 · 5 comments
Closed
Assignees
Labels
3 Story Point Estimate jan-mar qtr Possible FY Q4 candidate

Comments

@hannahramadan
Copy link
Contributor

hannahramadan commented Feb 13, 2024

Customers should be able to add custom attributes to a transaction via add_custom_attribute API. If a custom attribute is prefixed with llm., we should automatically attach it to the llm event, ie llm.<user_defined_metadata>

The conversation_id is now being handled like any other llm.* custom event and should be added to all events when available. Update the references to conversation_id to use this new llm. style.

Let's make sure the attributes are applied to all custom events. Not just chat completion summaries.

We should make sure this is captured in documentation and may need to additionally describe this functionality in the reserved keyword docs.

@workato-integration
Copy link

@kaylareopelle kaylareopelle moved this from Triage to To do in Ruby Engineering Board Feb 14, 2024
@kaylareopelle kaylareopelle changed the title Attach custom attributes to llm_events when prefixed with llm. OpenAI: Attach custom attributes to llm_events when prefixed with llm. Feb 14, 2024
@kford-newrelic kford-newrelic added the estimate Issue needing estimation label Feb 20, 2024
@kaylareopelle
Copy link
Contributor

While we're working on this, let's make sure the attributes are applied to all custom events. Not just chat completion summaries.

The conversation_id is now being handled like any other llm.* custom event and should be added to all events when available.

@kford-newrelic kford-newrelic added 3 Story Point Estimate and removed estimate Issue needing estimation labels Feb 21, 2024
@kaylareopelle kaylareopelle added the jan-mar qtr Possible FY Q4 candidate label Feb 22, 2024
@kaylareopelle
Copy link
Contributor

@hannahramadan - When you get back to this, let me know if you think removing the plumbing for conversation_id is an appropriate add for this ticket. Also happy to break it out into a different one.

@hannahramadan hannahramadan moved this from In Quarter to In Sprint in Ruby Engineering Board Feb 26, 2024
@hannahramadan
Copy link
Contributor Author

@kaylareopelle Yes let's throw that on here ◡̈

@hannahramadan
Copy link
Contributor Author

@kaylareopelle jsky Im going to remove conversation_id in a different PR. It'll touch a lot of files and I dont want that to detact from this addition!

@hannahramadan hannahramadan moved this from In Sprint to Code Complete/Done in Ruby Engineering Board Mar 4, 2024
@github-actions github-actions bot closed this as completed Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 Story Point Estimate jan-mar qtr Possible FY Q4 candidate
Projects
Archived in project
Development

No branches or pull requests

3 participants