fix(llama.cpp): disable mirostat as default

Even if increasing the quality of the output, it has shown to have performance drawbacks to be so noticeable that the confuses users about speed of LocalAI ( see also #2780 ). This changeset disables Mirostat by default (which can be still enabled manually). Signed-off-by: Ettore Di Giacinto <[email protected]>
mudler · Jul 18, 2024 · 4e40788 · 4e40788
1 parent 96f67ef
commit 4e40788
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/core/config/backend_config.go b/core/config/backend_config.go
@@ -261,7 +261,8 @@ func (cfg *BackendConfig) SetDefaults(opts ...ConfigLoaderOption) {
 	defaultTopP := 0.95
 	defaultTopK := 40
 	defaultTemp := 0.9
-	defaultMirostat := 2
+	// https://github.com/mudler/LocalAI/issues/2780
+	defaultMirostat := 0
 	defaultMirostatTAU := 5.0
 	defaultMirostatETA := 0.1
 	defaultTypicalP := 1.0