add readme for jamba

axolotl-ai-cloud · Mar 28, 2024 · a3cece5 · a3cece5
1 parent d437d2e
commit a3cece5
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/examples/jamba/README.md b/examples/jamba/README.md
@@ -0,0 +1,5 @@
+# Jamba
+
+qlora w/ deepspeed needs at least 2x GPUs and 35GiB VRAM per GPU
+
+qlora single-gpu - training will start, but loss is off by an order of magnitude