Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trace history instead of chat response in toolloop #39

Merged
merged 1 commit into from
Oct 24, 2024

Conversation

comhar
Copy link
Contributor

@comhar comhar commented Oct 21, 2024

This PR updates the tool loop tracing logic so that it traces history instead of the chat response. This change allows us to trace all requests to/from the ai as they happen. This should make it easier to debug issues with function calls and results. Previously only assistant responses were traced.

Here's what the order cancellation example from the docs looks like now:

tl_trace_clean.mov
FWIW, here's the tr_hist method from the video
def unpack_kwargs(kwargs): return ', '.join(f'{k}={v}' for k, v in kwargs.items())
    
def tr_hist(h):
    if not isinstance(h, list): h = [h]
    for o in h:
        role, content = o["role"], o["content"]
        for m in content:
            if m['type'] == 'text': print(f"{role}: {m['text']}")
            if m['type'] == 'tool_result': print(f"{role}: tool result: {m['content']}")
            if hasattr(m, 'text'): print(f"{role}: {m.text}")
            if hasattr(m, 'id'):
                print(f"\n================ tool call ================")
                print(f"{m.name}({unpack_kwargs(m.input)})")
                print(f"===========================================\n")
    print("\n")
See #9 for more context.

@comhar comhar added the enhancement New feature or request label Oct 21, 2024
Copy link

gitnotebooks bot commented Oct 21, 2024

Found 1 changed notebook. Review the changes at https://app.gitnotebooks.com/AnswerDotAI/claudette/pull/39

@comhar comhar changed the title trace history instead of chat response trace history instead of chat response in toolloop Oct 21, 2024
@jph00
Copy link
Contributor

jph00 commented Oct 24, 2024

Great thanks @comhar ! BTW don't forget to at-mention me or add me as a reviewer otherwise I only notice these things if/when I happen past them :)

@jph00 jph00 merged commit 3c9c452 into main Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants