Skip to content

Commit

Permalink
updated and reformatted README to include link to bearer auth use case
Browse files Browse the repository at this point in the history
  • Loading branch information
Salman Paracha authored and Salman Paracha committed Feb 10, 2025
1 parent 0ea237f commit f155704
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 53 deletions.
45 changes: 28 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,40 @@
<p align="center">
<div align="center">
<img src="docs/source/_static/img/arch-logo.png" alt="Arch Logo" width="75%" heigh=auto>
</p>
<p align="center">
<a href="https://www.producthunt.com/posts/arch-3?embed=true&utm_source=badge-top-post-badge&utm_medium=badge&utm_souce=badge-arch&#0045;3" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/top-post-badge.svg?post_id=565761&theme=light&period=daily" alt="Arch - Build&#0032;fast&#0044;&#0032;hyper&#0045;personalized&#0032;agents&#0032;with&#0032;intelligent&#0032;infra | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
</p>

Arch is an **intelligent (edge and LLM) proxy designed for agentic applications** - to help you protect, observe, and build agentic tasks by simply connecting (existing) APIs.

Built by the contributors of [Envoy Proxy](https://www.envoyproxy.io/) with the belief that:

>Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – outside core business logic.*
</div>
<div align="center">

[![pre-commit](https://github.com/katanemo/arch/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/katanemo/arch/actions/workflows/pre-commit.yml)
[![rust tests (prompt and llm gateway)](https://github.com/katanemo/arch/actions/workflows/rust_tests.yml/badge.svg)](https://github.com/katanemo/arch/actions/workflows/rust_tests.yml)
[![e2e tests](https://github.com/katanemo/arch/actions/workflows/e2e_tests.yml/badge.svg)](https://github.com/katanemo/arch/actions/workflows/e2e_tests.yml)
[![Build and Deploy Documentation](https://github.com/katanemo/arch/actions/workflows/static.yml/badge.svg)](https://github.com/katanemo/arch/actions/workflows/static.yml)

Arch is engineered with purpose-built LLMs to handle critical but undifferentiated tasks related to the handling and processing of prompts. This includes detecting and rejecting [jailbreak](https://github.com/verazuo/jailbreak_llms) attempts, intelligent task routing for improved accuracy, mapping user request into "backend" functions, and managing the observability of prompts and LLM API calls in a centralized way.
_Arch is an **intelligent (edge and LLM) proxy designed for agentic applications** - to help you protect, observe, and build agentic tasks by simply connecting (existing) APIs._

[Quickstart](##quickstart)
[Demos](##demos)
[Build Agentic App With Arch](###Build-AI-Agent-with-Arch-Gateway)
[Use Arch as a LLM Router](###Use-Arch-Gateway-as-LLM-Router-)
[Documentation](https://docs.archgw.com)
[Contact](##contact)
</div>

<a href="https://www.producthunt.com/posts/arch-3?embed=true&utm_source=badge-top-post-badge&utm_medium=badge&utm_souce=badge-arch&#0045;3" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/top-post-badge.svg?post_id=565761&theme=light&period=daily" alt="Arch - Build&#0032;fast&#0044;&#0032;hyper&#0045;personalized&#0032;agents&#0032;with&#0032;intelligent&#0032;infra | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>


Arch Gateway was built by the contributors of [Envoy Proxy](https://www.envoyproxy.io/) with the belief that:

>Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – outside core business logic.*

Arch is engineered with purpose-built LLMs to handle critical but pesky tasks related to the handling and processing of prompts. This includes detecting and rejecting [jailbreak](https://github.com/verazuo/jailbreak_llms) attempts, intent-based routing for improved task accuracy, mapping user request into "backend" functions, and managing the observability of prompts and LLM API calls in a centralized way.

**Core Features**:
- Built on [Envoy](https://envoyproxy.io): Arch runs alongside application servers as a separate containerized process, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs.
- Task Routing & Fast Function Calling. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts to build more task-accurate agentic applications.

- Intent-based prompt routing & fast ⚡ function-calling via APIs. Engineered with purpose-built [LLMs](https://huggingface.co/collections/katanemo/arch-function-66f209a693ea8df14317ad68) to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts to build more task-accurate agentic applications.
- Prompt [Guard](https://huggingface.co/collections/katanemo/arch-guard-6702bdc08b889e4bce8f446d): Arch centralizes guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code.
- Routing & Traffic Management: Arch centralizes calls to LLMs used by your applications, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability.
- LLM Routing & Traffic Management: Arch centralizes calls to LLMs used by your applications, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability.
- Observability: Arch uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance.
- Built on [Envoy](https://envoyproxy.io): Arch runs alongside application servers as a separate containerized process, and builds on top of Envoy's proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs.

**High-Level Sequence Diagram**:
![alt text](docs/source/_static/img/arch_network_diagram_high_level.png)
Expand All @@ -38,8 +48,9 @@ Arch is engineered with purpose-built LLMs to handle critical but undifferentiat
To get in touch with us, please join our [discord server](https://discord.gg/pGZf2gcwEc). We will be monitoring that actively and offering support there.

## Demos
* [Weather Forecast](demos/samples_python/weather_forecast/README.md) - Walk through of the core function calling capabilities of arch gateway using weather forecasting service
* [Network Agent](demos/samples_python/network_switch_operator_agent/README.md) - Build a networking co-pilot/agent agent with Arch
* [Sample App: Weather Forecast Agent](demos/samples_python/weather_forecast/README.md) - Walk through of the core function calling capabilities of arch gateway using weather forecasting service
* [Sample App: Network Operator Agent](demos/samples_python/network_switch_operator_agent/README.md) - Build a networking co-pilot/agent agent with Arch
* [User Cases: Connecting to SaaS APIs](demos/use_cases/spotify_bearer_auth) - Connect 3rd party SaaS APIs to your agentic chat experience.

## Quickstart

Expand Down
35 changes: 1 addition & 34 deletions demos/use_cases/spotify_bearer_auth/arch_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ system_prompt: |
Make sure your output is valid Markdown. And don't say "formatted in Markdown". Thanks!
llm_providers:
- name: OpenAI
- name: openai
provider_interface: openai
access_key: $OPENAI_API_KEY
model: gpt-4o
Expand Down Expand Up @@ -120,36 +120,3 @@ prompt_targets:
path: /v1/artists/{artist_id}/top-tracks
http_headers:
Authorization: "Bearer $SPOTIFY_CLIENT_KEY"

# - name: search_for_item
# description: Get information about albums, artists, playlists, tracks, shows, episodes or audiobooks. You can search for an item by its name, creator, or topic.
# parameters:
# - name: q
# description: Your search query, which can include keywords related to the item name, its creator, or its topic.
# required: true
# type: str
# - name: type
# type: str
# description: The type of the item to search for (e.g., album, artist, playlist, track, show, episode, audiobook).
# enum:
# - album
# - artist
# - playlist
# - track
# - show
# - episode
# - audiobook
# required: true
# - name: market
# type: str
# description: A country code
# default: US
# - name: limit
# type: integer
# description: The maximum number of results to return
# default: "5"
# endpoint:
# name: spotify
# path: /v1/search
# http_headers:
# Authorization: "Bearer $SPOTIFY_CLIENT_KEY"
4 changes: 2 additions & 2 deletions demos/use_cases/spotify_bearer_auth/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
services:
chatbot_ui:
build:
context: ../shared/chatbot_ui
context: ../../shared/chatbot_ui
ports:
- "18080:8080"
environment:
Expand All @@ -14,7 +14,7 @@ services:

jaeger:
build:
context: ../shared/jaeger
context: ../../shared/jaeger
ports:
- "16686:16686"
- "4317:4317"
Expand Down

0 comments on commit f155704

Please sign in to comment.