fix(llama.cpp): disable mirostat as default

Even if increasing the quality of the output, it has shown to have performance drawbacks to be so noticeable that the confuses users about speed of LocalAI. This changeset disables Mirostat by default (which can be still enabled manually). Signed-off-by: Ettore Di Giacinto <[email protected]>
mudler · Jul 18, 2024 · c680622 · c680622
1 parent 96f67ef
commit c680622
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/core/config/backend_config.go b/core/config/backend_config.go
@@ -261,7 +261,8 @@ func (cfg *BackendConfig) SetDefaults(opts ...ConfigLoaderOption) {
 	defaultTopP := 0.95
 	defaultTopK := 40
 	defaultTemp := 0.9
-	defaultMirostat := 2
+	// https://github.com/mudler/LocalAI/issues/2780
+	defaultMirostat := 0
 	defaultMirostatTAU := 5.0
 	defaultMirostatETA := 0.1
 	defaultTypicalP := 1.0