Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
277 changes: 163 additions & 114 deletions README.md

Large diffs are not rendered by default.

133 changes: 133 additions & 0 deletions docs/openai-responses.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
## OpenAI Responses API Support

This library now natively understands OpenAI's Responses API request shape and can emit Responses bodies when targeting the Responses endpoint. It remains fully stateless: it does not persist any conversation state; it simply translates requests to a universal format and back, passing through OpenAI state hints like `store` and `previous_response_id` unchanged.

### What’s supported

- **Request parsing**: Converts a Responses API request (`model`, `instructions`, `input`, etc.) into the universal request shape.
- **Emission to Responses**: Emits a valid Responses create body from the universal request.
- **State hints pass-through**: Preserves and forwards Responses state fields such as `store` and `previous_response_id` without altering them.
- **Tools**:
- Custom function tools map to/from `universal.tools` (JSON Schema based).
- Built-in tools (e.g., `web_search_preview`, `file_search`, `code_interpreter`, etc.) are preserved and round-tripped via `provider_params.responses_builtin_tools`.
- **Streaming**: Universal `stream: true|false` maps to Responses `stream` with the same boolean.
- **Token limits**: Universal `max_tokens` maps to Responses `max_output_tokens` when emitting.

### Shape selection (when we emit Responses vs Chat)

- If your target URL includes `/v1/responses`, the handler automatically annotates the universal request so that the OpenAI formatter emits a **Responses** body.
- Alternatively, you can force Responses emission by setting `provider_params.openai_target = "responses"` on the universal request before calling `fromUniversal("openai", ...)`.

### Request field mapping (high level)

- **instructions** ↔ universal `system` (string)
- **input** ↔ universal `messages` (roles: `user` | `system` | `developer`)
- **max_output_tokens** ↔ universal `max_tokens`
- **temperature/top_p** ↔ universal `temperature` / `top_p`
- **tools**:
- `type: "function"` tools ↔ `universal.tools`
- built-in tools (e.g., `web_search_preview`) ↔ `universal.provider_params.responses_builtin_tools`
- **tool_choice**:
- `"auto" | "none" | "required"` ↔ same in universal `tool_choice`
- `{ type: "function", name: "..." }` ↔ universal `tool_choice = { name }`
- **state hints** (pass-through in `provider_params`):
- `store`, `previous_response_id`, `include`, `text`, `parallel_tool_calls`, `service_tier`, `truncation`, `background`, `user`, `metadata`

### Using the handler with Responses endpoint

When you call the universal handler against `/v1/responses`, it emits a Responses body and forwards `store` and `previous_response_id`:

```ts
import { handleUniversalRequest } from "llm-bridge"

async function editFunction(request) {
// Keep it stateless: just pass through state hints
return {
request: {
...request,
provider_params: {
...(request.provider_params ?? {}),
previous_response_id: "resp_123",
store: true,
},
},
contextModified: false,
}
}

const { response } = await handleUniversalRequest(
"https://api.openai.com/v1/responses",
{
model: "gpt-4o",
input: [
{
type: "message",
role: "user",
content: [{ type: "input_text", text: "Hello" }],
},
],
store: true,
},
{ Authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
"POST",
editFunction,
)
```

### Using the translators directly

You can work with the translators without the handler.

```ts
import { toUniversal, fromUniversal } from "llm-bridge"

// 1) Responses -> Universal
const responsesReq = {
model: "gpt-4o",
instructions: "You are helpful.",
input: [
{
type: "message",
role: "user",
content: [
{ type: "input_text", text: "Hi" },
{
type: "input_image",
detail: "auto",
image_url: "https://example.com/image.png",
},
],
},
],
store: true,
previous_response_id: "resp_abc",
tools: [
{ type: "function", name: "get_weather", parameters: { type: "object" } },
{ type: "web_search_preview" }, // built-in tool
],
}

const universal = toUniversal("openai", responsesReq)

// 2) Universal -> Responses
// - Built-in tools are preserved via provider_params.responses_builtin_tools
// - state hints are passed through
const emitted = fromUniversal("openai", {
...universal,
provider_params: {
...(universal.provider_params ?? {}),
openai_target: "responses", // force Responses emission if needed
},
})
```

### Built-in tools vs custom function tools

- Custom function tools are always available on `universal.tools` with their JSON Schema `parameters`.
- Built-in tools are available in `universal.provider_params.responses_builtin_tools` and round-trip back to Responses `tools`.

### Notes

- The library remains stateless: it never stores Responses; it only forwards `store` and `previous_response_id` values from the request.
- You can continue using Chat Completions. The translator will emit Chat bodies unless the target URL is `/v1/responses` or you set `provider_params.openai_target = "responses"`.
- For more on the Responses API, see the official docs: [Responses API](https://platform.openai.com/docs/api-reference/responses).
21 changes: 11 additions & 10 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,30 @@ This directory contains practical examples demonstrating how to use LLM Bridge i
## 📂 Examples

### 🚀 Basic Examples

- **[basic-translation.ts](./basic-translation.ts)** - Simple provider-to-provider translation
- **[provider-detection.ts](./provider-detection.ts)** - Auto-detect provider format
- **[perfect-reconstruction.ts](./perfect-reconstruction.ts)** - Zero data loss round-trip conversion

### 🏗️ Advanced Integration Patterns

- **[universal-middleware.ts](./universal-middleware.ts)** - Universal LLM middleware for Express.js
- **[load-balancer.ts](./load-balancer.ts)** - Multi-provider load balancing with fallbacks
- **[cost-optimizer.ts](./cost-optimizer.ts)** - Automatic cost optimization across providers

### 🖼️ Multimodal Examples

- **[image-analysis.ts](./image-analysis.ts)** - Cross-provider image analysis
- **[multimodal-chat.ts](./multimodal-chat.ts)** - Multimodal chat application

### 🛠️ Tool Calling Examples

- **[function-calling.ts](./function-calling.ts)** - Tool calling across providers
- **[weather-agent.ts](./weather-agent.ts)** - Weather agent with tool calling

### 🔧 Utility Examples
- **[error-handling.ts](./error-handling.ts)** - Comprehensive error handling
- **[observability.ts](./observability.ts)** - Telemetry and monitoring
- **[token-counting.ts](./token-counting.ts)** - Token usage estimation

- (See repository tests and README for additional patterns)

### 🏢 Production Examples

- **[chatbot-service.ts](./chatbot-service.ts)** - Production chatbot service
- **[api-proxy.ts](./api-proxy.ts)** - Universal LLM API proxy
- **[batch-processor.ts](./batch-processor.ts)** - Batch processing with multiple providers

## 🚀 Running Examples

Expand All @@ -42,11 +40,14 @@ npm install
npx tsx examples/basic-translation.ts
npx tsx examples/load-balancer.ts
npx tsx examples/chatbot-service.ts
npx tsx examples/cost-optimizer.ts
npx tsx examples/function-calling.ts
npx tsx examples/image-analysis.ts
```

## 📝 Notes

- All examples are fully typed with TypeScript
- Examples use mock API calls for demonstration purposes
- Replace mock functions with actual provider SDK calls in production
- Each example includes comprehensive comments explaining the concepts
- Each example includes comprehensive comments explaining the concepts
30 changes: 13 additions & 17 deletions examples/basic-translation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,19 @@
* using LLM Bridge's core translation functions.
*/

import { toUniversal, fromUniversal, translateBetweenProviders } from '../src'
s
import { toUniversal, fromUniversal, translateBetweenProviders, OpenAIChatBody } from '../src'
// Example 1: OpenAI to Universal to Anthropic
console.log('🔄 Example 1: OpenAI → Universal → Anthropic')

const openaiRequest = {
const openaiRequest: OpenAIChatBody = {
model: "gpt-4",
messages: [
{ role: "system", content: "You are a helpful AI assistant" },
{ role: "user", content: "Explain quantum computing in simple terms" }
],
temperature: 0.7,
max_tokens: 500
} as any
}

console.log('📝 Original OpenAI Request:')
console.log(JSON.stringify(openaiRequest, null, 2))
Expand All @@ -40,14 +39,14 @@ console.log(JSON.stringify(anthropicRequest, null, 2))
// Example 2: Direct provider-to-provider translation
console.log('\n\n🚀 Example 2: Direct Translation (OpenAI → Google)')

const googleRequest = translateBetweenProviders("openai", "google", openaiRequest as any)
const googleRequest = translateBetweenProviders("openai", "google", openaiRequest)
console.log('🔍 Google Gemini Format:')
console.log(JSON.stringify(googleRequest, null, 2))

// Example 3: Multimodal content translation
console.log('\n\n🖼️ Example 3: Multimodal Content Translation')

const multimodalRequest = {
const multimodalRequest: OpenAIChatBody = {
model: "gpt-4-vision-preview",
messages: [{
role: "user",
Expand All @@ -62,35 +61,32 @@ const multimodalRequest = {
}
]
}]
} as any
}

const multimodalAnthropic = translateBetweenProviders("openai", "anthropic", multimodalRequest as any)
const multimodalAnthropic = translateBetweenProviders("openai", "anthropic", multimodalRequest)
console.log('🖼️ Multimodal Anthropic Format:')
console.log(JSON.stringify(multimodalAnthropic, null, 2))

// Example 4: Perfect reconstruction
console.log('\n\n♻️ Example 4: Perfect Reconstruction (Zero Data Loss)')

const originalRequest = {
const originalRequest: OpenAIChatBody = {
model: "gpt-4-turbo-preview",
messages: [
{ role: "user", content: "Hello world" }
],
temperature: 0.5,
response_format: { type: "json_object" },
seed: 12345,
top_p: 0.9
} as any
}

const universalFormat = toUniversal("openai", originalRequest as any)
const universalFormat = toUniversal("openai", originalRequest)
const reconstructed = fromUniversal("openai", universalFormat)

console.log('✅ Perfect reconstruction verified:')
console.log('Original === Reconstructed:', JSON.stringify(originalRequest) === JSON.stringify(reconstructed))
console.log('All OpenAI-specific fields preserved:',
reconstructed.response_format &&
reconstructed.seed === 12345 &&
reconstructed.top_p === 0.9
console.log('OpenAI fields preserved:',
reconstructed.top_p === 0.9 &&
reconstructed.temperature === 0.5
)

console.log('\n🎉 Basic translation examples completed!')
Loading
Loading