Is there a way to print both tool use requests and responses inside chat.toolloop?

[maraoz]

Hi, sorry if this is a dumb question but I've been trying to make this work and reading the source code and I've had no luck.

I'm playing with an assistant which has 3 tools.
I'm trying to print everything that's happening so I can debug the system prompt and the initial message prompt.
However, if I call chat.toolloop like so:

answer = chat.toolloop(main_prompt, trace_func=print)

I only get prints of:

• (1) the messages from the assistant before calling the tool
• (2) the tool_use request
• (3) the messages from the assistant after calling the tool and seeing the results

I'm not seeing, however, any output regarding the tool function call, nor the results and how they are sent to Claude (which would be very useful in debugging).

To work around this, I added prints inside the tool function, but the result is a little weird (at least to me):
The prints from the tool function call appear before (1) from above.

The question is: is there easy way to have the debug prints in chronological order (that is, (1), (2), tool function prints, (3))?
Thanks in advance! Claudette inspired me to play with Claude API and it's fun! :)

[jph00]

I want that too! At some point I'll get around to adding it...

[maraoz]

In case anyone is in the same situation...
For now I'm using the following decorator:

import functools

def debug_print(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        print(f"TOOL DEBUG: {func.__name__}() returned {result}")
        return result
    return wrapper

You can use it by prefixing your tool functions with @debug_print like so:

@debug_print
def get_customer_info(...):
  ...

If I have time this week I'll try to add the functionality to claudette (although I wouldn't do it with a decorator)

[comhar]

Printing the chat history (chat.h) is pretty useful for debugging. When running the order cancellation example from the docs, here's the full chat history which includes the results of the tool function calls.

[
 {'role': 'user', 'content': [{'type': 'text', 'text': 'Please cancel all orders for customer C1 for me.'}]},
 {'role': 'assistant', 'content': [TextBlock(text="Okay, let's cancel all orders for customer C1:", type='text'), ToolUseBlock(id='toolu_01YANwcUTV41qShFRHkqPXFn', input={'customer_id': 'C1'}, name='get_customer_info', type='tool_use')]},
 {'role': 'user',  'content': [{'type': 'tool_result', 'tool_use_id': 'toolu_01YANwcUTV41qShFRHkqPXFn', 'content': "{'name': 'John Doe', 'email': '[email protected]', 'phone': '123-456-7890', 'orders': [{'id': 'O1', 'product': 'Widget A', 'quantity': 2, 'price': 19.99, 'status': 'Cancelled'}, {'id': 'O2', 'product': 'Gadget B', 'quantity': 1, 'price': 49.99, 'status': 'Cancelled'}]}"}]},
 {'role': 'assistant', 'content': [TextBlock(text="Based on the customer information, it looks like there are 2 orders for customer C1:\n- Order O1 for Widget A (2 units)\n- Order O2 for Gadget B (1 unit)\n\nLet's go ahead and cancel both of these orders:", type='text'), ToolUseBlock(id='toolu_016NBwPF2pJvQFcQCeXn6h1s', input={'order_id': 'O1'}, name='cancel_order', type='tool_use')]},
 {'role': 'user', 'content': [{'type': 'tool_result', 'tool_use_id': 'toolu_016NBwPF2pJvQFcQCeXn6h1s', 'content': 'True'}]},
 {'role': 'assistant', 'content': [ToolUseBlock(id='toolu_01Fp5yy7apAKG2oXc2muX3Mz', input={'order_id': 'O2'}, name='cancel_order', type='tool_use')]},
 {'role': 'user', 'content': [{'type': 'tool_result', 'tool_use_id': 'toolu_01Fp5yy7apAKG2oXc2muX3Mz', 'content': 'True'}]},
 {'role': 'assistant', 'content': [TextBlock(text='Both order cancellations were successful. All orders for customer C1 have now been cancelled.', type='text')]}
]

[comhar]

@maraoz does printing the chat history (mentioned here) solve your issue?

[maraoz]

@comhar let me check and report back

[maraoz]

Not really. I'm using chat.toolloop (see: https://github.com/maraoz/claudette-scheduling-agent/blob/main/main.py#L576) and what I want is to be able to print function calls and results as they happen. From what I understand, your suggestion would mean the whole chat is printed at the end. Right?
If you find a way of using chat.h to achieve what I want let me know, I'm interested!

[comhar]

Not really. I'm using chat.toolloop (see: https://github.com/maraoz/claudette-scheduling-agent/blob/main/main.py#L576) and what I want is to be able to print function calls and results as they happen. From what I understand, your suggestion would mean the whole chat is printed at the end. Right? If you find a way of using chat.h to achieve what I want let me know, I'm interested!

Ah ok. If we change toolloop to trace history (self.h) instead of the Chat response we can generate something like this when using print as trace_func

raw trace_func

tl_trace_raw.mov

Here's a cleaner version which better highlights the tool call.

clean trace_func

tl_trace_clean.mov

@maraoz would this solve your problem?

[maraoz]

@comhar yes! love it :)