Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

App crush when loading falcon 3b abliterated. SmolChat-v0.0.3 #20

Closed
Tom-Neverwinter opened this issue Dec 30, 2024 · 2 comments
Closed
Assignees
Labels
bug Something isn't working

Comments

@Tom-Neverwinter
Copy link

Tom-Neverwinter commented Dec 30, 2024

Model: Falcon3-3B-Instruct-abliterated-Q6_K_L
App: SmolChat-v0.0.3
Phone: Pixel 9 pro XL
OS: (Android 15)

https://huggingface.co/bartowski/Falcon3-3B-Instruct-abliterated-GGUF/blob/main/Falcon3-3B-Instruct-abliterated-Q6_K_L.gguf

downloaded then loaded in app. it began to copy then crashed.

anytime the app is loaded it crashes.

testing uninstall then re-install [uninstalling makes the app work again .now testing the model after a re-download, sadly it still crashes]
same with this one: https://huggingface.co/bartowski/Falcon3-3B-Instruct-abliterated-GGUF/blob/main/Falcon3-3B-Instruct-abliterated-IQ4_NL.gguf

@shubham0204 shubham0204 self-assigned this Dec 31, 2024
@shubham0204
Copy link
Owner

The updated v0.0.3 release should fix the problem. It required me to sync the llama.cpp module with the upstream repository, which had integrated the features to support Falcon 3B. The PR ggerganov/llama.cpp#10883 has been merged in the master branch of llama.cpp.

Please confirm that the new release fixes the issue. We can close this issue then.

@shubham0204 shubham0204 reopened this Dec 31, 2024
@shubham0204 shubham0204 added the bug Something isn't working label Dec 31, 2024
@Tom-Neverwinter
Copy link
Author

hey it works!

your the best, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants