[no merge]Litellm zscsaler ai guard plugin

Dockerfile

@@ -12,7 +12,7 @@ WORKDIR /app
 USER root
 
 # Install build dependencies
-RUN apk add --no-cache gcc python3-dev openssl openssl-dev
+RUN apk add --no-cache --no-check-certificate gcc python3-dev openssl openssl-dev
 
 
 RUN pip install --upgrade pip && \
@@ -51,7 +51,7 @@ FROM $LITELLM_RUNTIME_IMAGE AS runtime
 USER root
 
 # Install runtime dependencies
-RUN apk add --no-cache openssl tzdata
+RUN apk add --no-cache --no-check-certificate  openssl tzdata
 
 WORKDIR /app
 # Copy the current directory contents into the container at /app

docker-compose.yml

@@ -7,10 +7,10 @@ services:
     image: ghcr.io/berriai/litellm:main-stable
     #########################################
     ## Uncomment these lines to start proxy with a config.yaml file ##
-    # volumes:
-    #  - ./config.yaml:/app/config.yaml <<- this is missing in the docker-compose file currently
-    # command:
-    #  - "--config=/app/config.yaml"
+    volumes:
+     - ./config.yaml:/app/config.yaml  #<<- this is missing in the docker-compose file currently
+    command:
+     - "--config=/app/config.yaml"
     ##############################################
     ports:
       - "4000:4000" # Map the container port to the host, change the host port if necessary

docs/my-website/docs/proxy/guardrails/zscaler_ai_guard.md

@@ -0,0 +1,125 @@
+# Zscaler AI Guard
+
+## Overview
+Zscaler AI guard enforces security policies for all traffic towards AI sites, models and applications. The AI guard is part of the Zero Trust Exchange and provides a comprehensive platform for visibility, control and deep packet inspection of AI prompts.
+
+## 1. Setup guardrails policy on Zscaler AI Guard
+Setup guardrails policy on Zscaler AI Guard, and get your ZSCALER_AI_GUARD_API_KEY, ZSCALER_AI_GUARD_POLICY_ID
+
+## 2. Define Zscaler AI Guard in `config.yaml`
+
+You can define Zscaler AI Guard settings directly in your LiteLLM `config.yaml` file.
+
+### Example Configuration: 
+Set ZSCALER_AI_GUARD_API_KEY, ZSCALER_AI_GUARD_POLICY_ID, ZSCALER_AI_GUARD_URL as enviroment variables
+
+```yaml
+guardrails:
+  - guardrail_name: "zscaler-ai-guard-during-guard"
+    litellm_params:
+      guardrail: zscaler_ai_guard
+      mode: "during_call"                  
+      api_key: os.environ/ZSCALER_AI_GUARD_API_KEY
+      api_base: os.environ/ZSCALER_AI_GUARD_URL  
+      policy_id: os.environ/ZSCALER_AI_GUARD_POLICY_ID
+  - guardrail_name: "zscaler-ai-guard-post-guard"
+    litellm_params:
+      guardrail: zscaler_ai_guard
+      mode: "post_call"                   
+      api_key: os.environ/ZSCALER_AI_GUARD_API_KEY
+      api_base: os.environ/ZSCALER_AI_GUARD_URL  
+      policy_id: os.environ/ZSCALER_AI_GUARD_POLICY_ID
+```
+
+## 3. Test request 
+
+Expect this to fail since since `[email protected]` in the request is PII
+
+```shell
+curl -i http://localhost:4000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
+  -d '{
+    "model": "gpt-3.5-turbo",
+    "messages": [
+      {"role": "user", "content": "hi my email is [email protected]"}
+    ],
+    "guardrails": ["zscaler-ai-guard-during-guard", "zscaler-ai-guard-post-guard"]
+  }'
+```
+
+## 4. Behavior on Violations
+
+### Prompt is Blocked
+When input violates Zscaler AI Guard policies, it returns:
+- **HTTP Status**: 400
+- **Error Type**: `Guardrail Policy Violation`
+- **blocking_info**: 
+   - `transactionId`: Zscaler AI Guard transactionId for debugging
+   - `message`: Prompt or LLM response is blocked
+   - `blockingDetectors`: the list of Zscaler AI Guard detectors that block the request
+
+#### Example Response
+```json
+{
+   "error": {
+      "error_type": "Guardrail Policy Violation",
+      "blocking_info": {
+         "transactionId": "1234abcd-5678-efgh-9101-ijklmnopqr",
+         "message": "Prompt violates Zscaler AI Guard policy.",
+         "blockingDetectors": ["toxicity"]
+      }
+   },
+   "type": "None",
+   "param": "None",
+   "code": "400"
+}
+```
+
+### LLM response Blocked
+When output violates Zscaler AI Guard policies, it returns:
+- **HTTP Status**: 400
+- **Error Type**: `Guardrail Policy Violation`
+- **blocking_info**: 
+   - `transactionId`: Zscaler AI Guard transactionId for debuging
+   - `message`: Prompt or LLM response is blocked
+   - `blockingDetectors`: the list of Zscaler AI Guard detectors that block the request
+
+#### Example Response
+```json
+{
+   "error": {
+      "error_type": "Guardrail Policy Violation",
+      "blocking_info": {
+         "transactionId": "5678abcd-9101-efgh-1234-ijklmnopqr",
+         "message": "LLM response violates Zscaler AI Guard policy.",
+         "blockingDetectors": ["toxicity"]
+      }
+   },
+   "type": "None",
+   "param": "None",
+   "code": "400"
+}
+```
+
+
+## 5. Error Handling for Service Issues
+
+In cases where Zscaler AI Guard encounters operational issues, it returns:
+- **HTTP Status**: 500
+- **Error Type**: `Guardrail Service Operational Issue`
+- **reason**: the detailed reason 
+
+#### Example Response
+```json
+{
+   "error": {
+      "error_type": "Zscaler AI Guard Service Operational Issue",
+      "reason": "Action field in response is None, expecting 'ALLOW', 'BLOCK' or 'DETECT."
+   },
+   "type": "None",
+   "param": "None",
+   "code": "500"
+}
+```
+

litellm/proxy/guardrails/guardrail_hooks/zscaler_ai_guard/__init__.py

@@ -0,0 +1,33 @@
+from typing import TYPE_CHECKING
+
+from litellm.types.guardrails import SupportedGuardrailIntegrations
+
+from .zscaler_ai_guard import ZscalerAIGuard
+
+if TYPE_CHECKING:
+    from litellm.types.guardrails import Guardrail, LitellmParams
+
+
+def initialize_guardrail(litellm_params: "LitellmParams", guardrail: "Guardrail"):
+    import litellm
+
+    _zscaler_ai_guard_callback = ZscalerAIGuard(
+        api_base=litellm_params.api_base,
+        api_key=litellm_params.api_key,
+        guardrail_name=guardrail.get("guardrail_name", ""),
+        event_hook=litellm_params.mode,
+        default_on=litellm_params.default_on,
+    )
+    litellm.logging_callback_manager.add_litellm_callback(_zscaler_ai_guard_callback)
+
+    return _zscaler_ai_guard_callback
+
+
+guardrail_initializer_registry = {
+    SupportedGuardrailIntegrations.ZSCALER_AI_GUARD.value: initialize_guardrail,
+}
+
+
+guardrail_class_registry = {
+    SupportedGuardrailIntegrations.ZSCALER_AI_GUARD.value: ZscalerAIGuard,
+}