Skip to content

Malformed JSON response while result from LLM call looks good #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
qbloq opened this issue Apr 3, 2025 · 5 comments
Open

Malformed JSON response while result from LLM call looks good #191

qbloq opened this issue Apr 3, 2025 · 5 comments

Comments

@qbloq
Copy link

qbloq commented Apr 3, 2025

Hey there! Thanks for this library.

I'm running an agent with debug: true. The first part of the output I provide here is the tail of the debug output which confirms that the LLM could predict the Agent's outputs right (Intent and Scheduling Action); however the transformation to JSON is having issues.

AGENT CALL:

    const intentExt = await customerIntentExtraction.forward(ai, {
      context, message
    }, { stream: false, debug: true })

    console.log(intentExt)

OUTPUT:

Assistant:
Intent: scheduling
Scheduling Action: inquiry

{ intent: "scheduling\nScheduling Action: inquiry" }

I'm running latest: 11.0.33

@dosco
Copy link
Collaborator

dosco commented Apr 5, 2025

looking into it

dosco added a commit that referenced this issue Apr 5, 2025
@qbloq
Copy link
Author

qbloq commented Apr 7, 2025

Not seeing more of this issue in my workflows, thanks!

I'm seeing this other incorrect parsing issue on arrays, but not being a blocker.

Assistant:
...
Dog Colors: ["Gris", "Blanco", "Negro"]
...

{
  ...
  dogColors: [ "Gris", "Blanco", "Negro", "Blanco", "Negro", "Negro" ],
  ...
}

@dosco
Copy link
Collaborator

dosco commented Apr 7, 2025

Can you create a new issue for this also what model are you seeing this with seems like the model is trying to create a json object instead of just a json array.

@qbloq
Copy link
Author

qbloq commented Apr 7, 2025

Thanks, will create a new ticket for that one.

As to this very issue, it's appearing again:

Assistant:
Intent: dog_info
Dog Name: luki
Dog Age: 2 meses
Dog Breed: golden

{
  intent: "dog_info",
  dogName: "luki\nDog Age: 2 meses",
  dogBreed: "golden"
}

It breaks when there're more than 3 outputs

@dosco
Copy link
Collaborator

dosco commented Apr 13, 2025

thanks will take a second look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants