Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(model): add rwkv-6-world-7b to the gallery #4270

Merged
merged 1 commit into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions gallery/index.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,27 @@
---
- &rwkv
url: "github:mudler/LocalAI/gallery/rwkv.yaml@master"
name: "rwkv-6-world-7b"
license: apache-2.0
urls:
- https://huggingface.co/RWKV/rwkv-6-world-7b
- https://huggingface.co/bartowski/rwkv-6-world-7b-GGUF
tags:
- llm
- rwkv
- cpu
- gpu
- rnn
description: |
RWKV (pronounced RwaKuv) is an RNN with GPT-level LLM performance, and can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7.
So it's combining the best of RNN and transformer - great performance, fast inference, fast training, saves VRAM, "infinite" ctxlen, and free text embedding. Moreover it's 100% attention-free, and a Linux Foundation AI project.
overrides:
parameters:
model: rwkv-6-world-7b-Q4_K_M.gguf
files:
- filename: rwkv-6-world-7b-Q4_K_M.gguf
sha256: f74574186fa4584f405e92198605680db6ad00fd77974ffa14bf02073bb90273
uri: huggingface://bartowski/rwkv-6-world-7b-GGUF/rwkv-6-world-7b-Q4_K_M.gguf
- &qwen25coder
name: "qwen2.5-coder-14b"
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
Expand Down
23 changes: 23 additions & 0 deletions gallery/rwkv.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
name: "rwkv"

config_file: |
parameters:
top_k: 80
temperature: 0.9
max_tokens: 4098
top_p: 0.8
context_size: 4098

roles:
user: "User: "
system: "System: "
assistant: "Assistant: "

stopwords:
- 'Assistant:'

template:
chat: "{{.Input}}\nAssistant: "
completion: |
{{.Input}}
Loading