updating doc versions, images and cleaning up section for prompt-guard (

#320)

* updating doc versions, images and cleaning up section for prompt-guard

* updating based on feedback

---------

Co-authored-by: Salman Paracha <[email protected]>

README.md

@@ -1,4 +1,4 @@
-![alt text](image.png)
+![alt text](docs/source/_static/img/arch-logo.png)
 <a href="https://www.producthunt.com/posts/arch-3?embed=true&utm_source=badge-top-post-badge&utm_medium=badge&utm_souce=badge-arch&#0045;3" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/top-post-badge.svg?post_id=565761&theme=light&period=daily" alt="Arch - Build&#0032;fast&#0044;&#0032;hyper&#0045;personalized&#0032;agents&#0032;with&#0032;intelligent&#0032;infra | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
 
 
@@ -9,20 +9,20 @@
 
 ## Build fast, observable, and personalized AI agents.
 
-Arch is an intelligent [Layer 7](https://www.cloudflare.com/learning/ddos/what-is-layer-7/) distributed proxy designed to protect, observe, and personalize AI agents with your APIs.
+Arch is an intelligent [Layer 7](https://www.cloudflare.com/learning/ddos/what-is-layer-7/) gateway designed to protect, observe, and personalize AI agents with your APIs.
 
-Engineered with purpose-built LLMs, Arch handles the critical but undifferentiated tasks related to the handling and processing of prompts, including detecting and rejecting [jailbreak](https://github.com/verazuo/jailbreak_llms) attempts, intelligently calling "backend" APIs to fulfill the user's request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability of prompts and LLM interactions in a centralized way.
+Engineered with purpose-built LLMs, Arch handles the critical but undifferentiated tasks related to the handling and processing of prompts, including detecting and rejecting [jailbreak](https://github.com/verazuo/jailbreak_llms) attempts, intelligently calling "backend" APIs to fulfill the user's request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability of prompts and LLM API calls in a centralized way.
 
  Arch is built on (and by the core contributors of) [Envoy Proxy](https://www.envoyproxy.io/) with the belief that:
 
 >Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – all outside business logic.*
 
 **Core Features**:
-  - Built on [Envoy](https://envoyproxy.io): Arch runs alongside application servers, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs.
-  - Function Calling for fast Agentic and RAG apps. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts.
-  - Prompt [Guard](https://huggingface.co/collections/katanemo/arch-guard-6702bdc08b889e4bce8f446d): Arch centralizes prompt guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code.
-  - Traffic Management: Arch manages LLM calls, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability.
-  - Standards-based Observability: Arch uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance.
+  - Built on [Envoy](https://envoyproxy.io): Arch runs alongside application servers as a separate containerized process, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs.
+  - Function Calling for fast Agents and RAG apps. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts.
+  - Prompt [Guard](https://huggingface.co/collections/katanemo/arch-guard-6702bdc08b889e4bce8f446d): Arch centralizes guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code.
+  - Routing & Traffic Management: Arch manages LLM calls, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability.
+  - Observability: Arch uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance.
 
 **Jump to our [docs](https://docs.archgw.com)** to learn how you can use Arch to improve the speed, security and personalization of your GenAI apps.
 
@@ -107,20 +107,6 @@ prompt_targets:
           type: int
           description: The number of days for which to gather device statistics.
           default: "7"
-    - name: reboot_devices
-      description: Reboot a list of devices
-      endpoint:
-        name: app_server
-        path: /agent/device_reboot
-      parameters:
-        - name: device_ids
-          type: list
-          description: A list of device identifiers (IDs).
-          required: true
-        - name: days
-          type: int
-          description: A list of device identifiers (IDs)
-          default: "7"
 
 # Arch creates a round-robin load balancing between different endpoints, managed via the cluster subsystem.
 endpoints:
@@ -158,7 +144,7 @@ print("OpenAI Response:", response.choices[0].message.content)
 ```
 
 ### [Observability](https://docs.archgw.com/guides/observability/observability.html)
-Arch is designed to support best-in class observability by supporting open standards. Please read our [docs](https://docs.archgw.com/guides/observability/observability.html) on observability for more details on tracing, metrics, and logs
+Arch is designed to support best-in class observability by supporting open standards. Please read our [docs](https://docs.archgw.com/guides/observability/observability.html) on observability for more details on tracing, metrics, and logs. The screenshot below is from our integration with Signoz (among others)
 
 ![alt text](docs/source/_static/img/tracing.png)
 

docs/source/concepts/tech_overview/terminology.rst

@@ -33,7 +33,7 @@ hosted on a different network (an API call). But these two processes are conside
 work in building generative AI apps. Prompt targets are endpoints that receive prompts that are processed by Arch.
 For example, Arch enriches incoming prompts with metadata like knowing when a request is a follow-up or clarifying prompt
 so that you can build faster, more accurate retrieval (RAG) apps. To support agentic apps, like scheduling travel plans or
-sharing comments on a document - via prompts, Bolt uses its function calling abilities to extract critical information from
+sharing comments on a document - via prompts, Arch uses its function calling abilities to extract critical information from
 the incoming prompt (or a set of prompts) needed by a downstream backend API or function call before calling it directly.
 
 **Error Target**: :ref:`Error targets <error_target>` are those endpoints that receive forwarded errors from Arch when issues arise,

docs/source/conf.py

@@ -15,7 +15,7 @@
 project = "Arch Docs"
 copyright = "2024, Katanemo Labs, Inc"
 author = "Katanemo Labs, Inc"
-release = " v0.1.2"
+release = " v0.1.5"
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

docs/source/get_started/includes/quickstart.yaml

@@ -1,44 +1,43 @@
 version: v0.1
-
-listen:
-  address: 0.0.0.0 # or 127.0.0.1
-  port: 10000
-  # Defines how Arch should parse the content from application/json or text/pain Content-type in the http request
+listener:
+  address: 127.0.0.1
+  port: 8080 #If you configure port 443, you'll need to update the listener with tls_certificates
   message_format: huggingface
 
 # Centralized way to manage LLMs, manage keys, retry logic, failover and limits in a central way
 llm_providers:
   - name: OpenAI
     provider: openai
     access_key: $OPENAI_API_KEY
-    model: gpt-4o
+    model: gpt-3.5-turbo
     default: true
-    stream: true
 
 # default system prompt used by all prompt targets
-system_prompt: You are a network assistant that just offers facts; not advice on manufacturers or purchasing decisions.
+system_prompt: |
+  You are a network assistant that helps operators with a better understanding of network traffic flow and perform actions on networking operations. No advice on manufacturers or purchasing decisions.
 
 prompt_targets:
-  - name: reboot_devices
-    description: Reboot specific devices or device groups
-
-    path: /agent/device_reboot
-    parameters:
-      - name: device_ids
-        type: list
-        description: A list of device identifiers (IDs) to reboot.
-        required: false
-      - name: device_group
-        type: str
-        description: The name of the device group to reboot
-        required: false
+    - name: device_summary
+      description: Retrieve network statistics for specific devices within a time range
+      endpoint:
+        name: app_server
+        path: /agent/device_summary
+      parameters:
+        - name: device_ids
+          type: list
+          description: A list of device identifiers (IDs) to retrieve statistics for.
+          required: true  # device_ids are required to get device statistics
+        - name: days
+          type: int
+          description: The number of days for which to gather device statistics.
+          default: "7"
 
 # Arch creates a round-robin load balancing between different endpoints, managed via the cluster subsystem.
 endpoints:
   app_server:
     # value could be ip address or a hostname with port
     # this could also be a list of endpoints for load balancing
     # for example endpoint: [ ip1:port, ip2:port ]
-    endpoint: 127.0.0.1:80
+    endpoint: host.docker.internal:18083
     # max time to wait for a connection to be established
     connect_timeout: 0.005s

docs/source/guides/prompt_guard.rst

@@ -1,41 +1,26 @@
 .. _prompt_guard:
 
 Prompt Guard
-================
+=============
 
-**Prompt guard** is a security and validation layer designed to protect prompt-based systems by filtering and analyzing inputs before they reach the core processing stages.
-In applications where prompts generate responses or execute specific actions based on user inputs, prompt guard minimizes risks like malicious inputs, unexpected errors, or misaligned outputs.
-By adding a layer of input scrutiny, prompt guard ensures safer, more reliable, and accurate interactions in prompt-driven environments.
+**Prompt guard** is a security and validation feature offered in Arch to protect agents, by filtering and analyzing prompts before they reach your application logic.
+In applications where prompts generate responses or execute specific actions based on user inputs, prompt guard minimizes risks like malicious inputs (or misaligned outputs).
+By adding a layer of input scrutiny, prompt guards ensures safer, more reliable, and accurate interactions with agents.
 
 Why Prompt Guard
 ----------------
 
 .. vale Vale.Spelling = NO
 
-- **Input Validation**
-    - **Type Enforcement**: Ensures that inputs are of the expected data types, such as integers, strings, lists, or specific formats, reducing errors from unexpected data.
-    - **Value Constraints**: Restricts inputs to valid ranges, lengths, or patterns to avoid unusual or incorrect responses.
-
-- **Prompt Sanitization**
+- **Prompt Sanitization via Arch-Guard**
     - **Jailbreak Prevention**: Detects and filters inputs that might attempt jailbreak attacks, like alternating LLM intended behavior, exposing the system prompt, or bypassing ethnics safety.
 
-- **Intent Detection**
-    - **Behavioral Analysis**: Analyzes prompt intent to detect if the input aligns with the function’s intended use. This can help prevent unwanted behavior, such as attempts to bypass limitations or misuse system functions.
-
 - **Dynamic Error Handling**
     - **Automatic Correction**: Applies error-handling techniques to suggest corrections for minor input errors, such as typos or misformatted data.
     - **Feedback Mechanism**: Provides informative error messages to users, helping them understand how to correct input mistakes or adhere to guidelines.
 
-- **Policy Enforcement**
-    - **Role-Based Filtering**: Customizes input validation based on user roles or permissions, allowing more flexibility or stricter enforcement depending on user access.
-    - **Compliance Checks**: Ensures inputs meet compliance or regulatory standards, especially in fields like finance or healthcare, where prompt outputs must align with strict guidelines.
-
-
-Arch-Guard
-----------
-In the evolving landscape of LLM-powered applications, safeguarding against prompt attacks is crucial.
-These attacks involve malicious prompts crafted to manipulate the intended behavior of the model, potentially leading to undesirable outcomes.
-Arch-Guard is designed to address this challenge.
+.. Note::
+    Today, Arch offers support for jailbreak via Arch-Guard. We will be adding support for additional guards in Q1, 2025 (including response guardrails)
 
 What Is Arch-Guard
 ~~~~~~~~~~~~~~~~~~
@@ -60,36 +45,19 @@ How Arch-Guard Works
 
 #. **Pre-Processing Stage**
 
-    As a request or prompt is received, Prompt Guard first performs validation, applying any type, format, or constraint checks. If any violations are detected, the input is flagged, and a tailored error message may be returned.
-
-#. **Sanitization Stage**
-
-    The prompt is analyzed for potentially harmful or inappropriate content, and necessary filters are applied to clean the input.
-
-#. **Behavior Analysis**
-
-    Next, the system assesses the intent and context of the prompt, verifying that it aligns with predefined function requirements. If the prompt raises any red flags, it can be modified or flagged for review.
+    As a request or prompt is received, Arch Guard first performs validation. If any violations are detected, the input is flagged, and a tailored error message may be returned.
 
 #. **Error Handling and Feedback**
 
     If the prompt contains errors or does not meet certain criteria, the user receives immediate feedback or correction suggestions, enhancing usability and reducing the chance of repeated input mistakes.
 
-#. **Output Control**
-
-    After input validation and filtering, the prompt is allowed to proceed to the main processing phase. The output can also undergo a final check to ensure compliance with content guidelines or role-based policies.
-
-
-Benefits of Using Prompt Guard
+Benefits of Using Arch Guard
 ------------------------------
 
 - **Enhanced Security**: Protects against injection attacks, harmful content, and misuse, securing both system and user data.
 
-- **Increased Accuracy**: Filters out inappropriate or misaligned inputs, leading to more accurate and intended outputs.
-
 - **Better User Experience**: Clear feedback and error correction improve user interactions by guiding them to correct input formats and constraints.
 
-- **Regulatory Compliance**: Ensures that prompts adhere to necessary guidelines, especially for sensitive fields, minimizing the risk of regulatory breaches.
-
 
 Summary
 -------