Create token_count attribute
#2489
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This attribute is available on ChatCompletionMessage and Embedding
kaylareopelle
commented
Mar 9, 2024
| @@ -101,8 +102,7 @@ def create_chat_completion_messages(parameters, summary_id) | |||
| role: message[:role] || message['role'], | |||
| sequence: index, | |||
| completion_id: summary_id, | |||
| vendor: VENDOR, | |||
| is_response: true | |||
There was a problem hiding this comment.
This is an unrelated fix. This first group of messages are not from a response. They're from a request. The is_response should only be attached to the event if it has a truthy value.
kaylareopelle
commented
Mar 9, 2024
| end | ||
| end | ||
|
|
||
| def calculate_token_count(model, content) | ||
| # return unless NewRelic::Agent.config['ai_monitoring.record_content.enabled'] |
There was a problem hiding this comment.
This'll get uncommented once the config PR is merged
kaylareopelle
requested review from
fallwith,
hannahramadan and
tannalynn
as code owners
hannahramadan
approved these changes
Mar 13, 2024
SimpleCov Report
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
token_counttoChatCompletionMessageandEmbeddingtoken_countusing the customer-provided callback set byset_llm_token_count_callbackand assigns it in OpenAI instrumentationtoken_countattribute.Closes #2450