Deprecated functionality

Author: assouktim

README.md

@@ -43,6 +43,11 @@ HUGGING_FACE_HUB_TOKEN="<YOUR HF HUB TOKEN>"
 python -m llm_inference --model "cmarkea/bloomz-3b-retriever-v2" --task EMBEDDING
 ```
 
+The server is designed to run one task at a time. There are three different tasks:
+- EMBEDDING
+- SCORING
+- GUARDRAIL
+
 ### API Endpoints
 
 You can access server documentation through this endpoint : `/docs`

llm_inference/routes/guardrail.py

@@ -29,7 +29,7 @@ def inference(request: GuardrailRequest) -> GuardrailResponse:
     try:
         with metrics.BATCH_INFERENCE_TIME.time():
             outputs = ServerPipeline().pipeline(
-                request.text, function_to_apply="sigmoid", return_all_scores=True
+                request.text, function_to_apply="sigmoid", top_k=None
             )
     except Exception as e:
         metrics.REQUEST_FAILURE.inc()

llm_inference/routes/scoring.py

@@ -34,7 +34,7 @@ def inference(request: ScoringRequest) -> ScoringResponse:
                     for context in request.contexts
                 ],
                 function_to_apply="softmax",
-                return_all_scores=True,
+                top_k=None,
             )
     except Exception as e:
         metrics.REQUEST_FAILURE.inc()