Skip to content

Conversation

CharlieFRuan
Copy link
Member

@CharlieFRuan CharlieFRuan commented May 5, 2025

This PR adds Qwen3 models from WebLLM 0.2.79.

We also add a config enable_thinking that can be toggled in the settings page to decide whether the Qwen3 model should reason or not. By default, it is turned off. Turning it on will cause it to reason and will also change its temperature and top_p, following https://huggingface.co/Qwen/Qwen3-0.6B#best-practices

In addition, when we want to summarize/create a topic, we should never reason, as it would take way too long.

In addition, for multi-round chat, we remove the <think>thinking tokens here</think>\n\n from the history assistant's message.

@CharlieFRuan
Copy link
Member Author

CharlieFRuan commented May 5, 2025

@Neet-Nestor Hi Nestor! Could you review this PR when you get a chance? Currently, changing the setting page does not seem to be very robust. Specifically, I added an enableThinking checkbox, which when checked, we should see reasoning behavior in Qwen3 models. This only rarely works as expected for now. Thanks!

Edit: should be fixed now

@CharlieFRuan
Copy link
Member Author

Never mind, the settings toggling should work now. There was a bug in isDifferentConfig(): config[field] !== config[field] should be this.llmConfig[field] !== config[field], as fixed in the latest commit.

@Neet-Nestor
Copy link
Collaborator

Neet-Nestor commented May 5, 2025

@CharlieFRuan I made several changes:

  1. Put the thinking toggle button in model setting instead of global settings
  2. Add another thinking toggle button in chat actions toolbar
  3. Fix the issue of <think> </think> tags in chat titles
  4. Update store version number in config.ts

I have tested and verified these changes by Qwen-0.6b. In the first screenshot the thinking toggle is on while in the second it's off.

Screenshot 2025-05-05 at 1 08 57 AM Screenshot 2025-05-05 at 1 08 48 AM

@Neet-Nestor
Copy link
Collaborator

I made another change to remove the empty <think> </think> tags in non-thinking mode.

Screenshot 2025-05-05 at 1 17 55 AM

@Neet-Nestor Neet-Nestor merged commit ac629bc into mlc-ai:main May 5, 2025
@CharlieFRuan
Copy link
Member Author

@Neet-Nestor Thank you so much! These are great changes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants