Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update AI documentation - add tutorials, fix inconsistencies, add more examples #629

Merged
merged 5 commits into from
Nov 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions content/guides/box-ai/ai-agents/ai-agent-overrides.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
---
rank: 4
related_endpoints:
- get_ai_agent_default
- post_ai_text_gen
- post_ai_ask
related_guides:
- box-ai/ai-tutorials/prerequisites
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/default-agent-overrides
---

# Override AI model configuration

<Message type="notice">
Endpoints related to metadata extraction are currently a beta feature offered subject to Box’s Main Beta Agreement, and the available capabilities may change. Box AI API is available to all Enterprise Plus customers.
</Message>

The `ai_agent` configuration allows you to override the default AI model configuration. It is available for the following endpoints:

* [`POST ai/ask`][ask]
* [`POST ai/text_gen`][text-gen]
* [`POST ai/extract`][extract]
* [`POST ai/extract_structured`][extract-structured]

<Message type='tip'>

Use the [`GET ai_agent_default`][agent] endpoint to fetch the default configuration.

</Message>

The override examples include:

* Replacing the default AI model with a custom one based on your organization's needs.
* Tweaking the base `prompt` to allow a more customized user experience.
* Changing a parameter, such as `temperature`, to make the results more or less creative.

## Sample configuration

A complete configuration for `ai/ask` is as follows:

```sh
{
"type": "ai_agent_ask",
"basic_text": {
"llm_endpoint_params": {
"type": "openai_params",
"frequency_penalty": 1.5,
"presence_penalty": 1.5,
"stop": "<|im_end|>",
"temperature": 0,
"top_p": 1
},
"model": "azure__openai__gpt_3_5_turbo_16k",
"num_tokens_for_completion": 8400,
"prompt_template": "It is `{current_date}`, consider these travel options `{content}` and answer the `{user_question}`.",
"system_message": "You are a helpful travel assistant specialized in budget travel"
},
"basic_text_multi": {
"llm_endpoint_params": {
"type": "openai_params",
"frequency_penalty": 1.5,
"presence_penalty": 1.5,
"stop": "<|im_end|>",
"temperature": 0,
"top_p": 1
},
"model": "azure__openai__gpt_3_5_turbo_16k",
"num_tokens_for_completion": 8400,
"prompt_template": "It is `{current_date}`, consider these travel options `{content}` and answer the `{user_question}`.",
"system_message": "You are a helpful travel assistant specialized in budget travel"
},
"long_text": {
"embeddings": {
"model": "openai__text_embedding_ada_002",
"strategy": {
"id": "basic",
"num_tokens_per_chunk": 64
}
},
"llm_endpoint_params": {
"type": "openai_params",
"frequency_penalty": 1.5,
"presence_penalty": 1.5,
"stop": "<|im_end|>",
"temperature": 0,
"top_p": 1
},
"model": "azure__openai__gpt_3_5_turbo_16k",
"num_tokens_for_completion": 8400,
"prompt_template": "It is `{current_date}`, consider these travel options `{content}` and answer the `{user_question}`.",
"system_message": "You are a helpful travel assistant specialized in budget travel"
},
"long_text_multi": {
"embeddings": {
"model": "openai__text_embedding_ada_002",
"strategy": {
"id": "basic",
"num_tokens_per_chunk": 64
}
},
"llm_endpoint_params": {
"type": "openai_params",
"frequency_penalty": 1.5,
"presence_penalty": 1.5,
"stop": "<|im_end|>",
"temperature": 0,
"top_p": 1
},
"model": "azure__openai__gpt_3_5_turbo_16k",
"num_tokens_for_completion": 8400,
"prompt_template": "It is `{current_date}`, consider these travel options `{content}` and answer the `{user_question}`.",
"system_message": "You are a helpful travel assistant specialized in budget travel"
}
}
```

### Differences in parameter sets

The set of parameters available for `ask`, `text_gen`, `extract`, `extract_structured` differs slightly, depending on the API call.

* The agent configuration for the `ask` endpoint includes `basic_text`, `basic_text_multi`, `long_text` and `long_text_multi` parameters. This is because of the `mode` parameter you use to specify if the request is for a single item or multiple items. If you selected `multiple_item_qa` as the `mode`, you can also use `multi` parameters for overrides.

* The agent configuration for `text_gen` includes the `basic_gen` parameter
that is used to generate text.

### LLM endpoint params

The `llm_endpoint_params` configuration options differ depending on the overall AI model being [Google][google-params], [OpenAI][openai-params] or [AWS][aws-params] based.

For example, both `llm_endpoint_params` objects accept a `temperature` parameter, but the outcome differs depending on the model.

For Google and AWS models, the [`temperature`][google-temp] is used for sampling during response generation, which occurs when `top-P` and `top-K` are applied. Temperature controls the degree of randomness in the token selection.

For OpenAI models, [`temperature`][openai-temp] is the sampling temperature with values between 0 and 2. Higher values like 0.8 make the output more random, while lower values like 0.2 make it more focused and deterministic. When introducing your own configuration, use `temperature` or or `top_p` but not both.

### System message

The `system_message` parameter's aim is to help the LLM understand its role and what it’s supposed to do.
For example, if your solution is processing travel itineraries, you can add a system message saying:

```sh
You are a travel agent aid. You are going to help support staff process large amounts of schedules, tickets, etc.
```

This message is separate from the content you send in, but it can improve the results.

### Number of tokens for completion

The `num_tokens_for_completion` parameter represents the number of [tokens][openai-tokens] Box AI can return. This number can vary based on the model used.

[ask]: e://post_ai_ask#param_ai_agent
[text-gen]: e://post_ai_text_gen#param_ai_agent
[extract]: e://post_ai_extract#param_ai_agent
[extract-structured]: e://post_ai_extract_structured#param_ai_agent
[google-params]: r://ai-llm-endpoint-params-google
[openai-params]: r://ai-llm-endpoint-params-openai
[openai-tokens]: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them
[agent]: e://get_ai_agent_default
[google-temp]: https://ai.google.dev/gemini-api/docs/models/generative-models#model-parameters
[openai-temp]: https://community.openai.com/t/temperature-top-p-and-top-k-for-chatbot-responses/295542
[aws-params]: r://ai-llm-endpoint-params-aws
12 changes: 6 additions & 6 deletions content/guides/box-ai/ai-agents/ai-agent-versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ related_endpoints:
- post_ai_text_gen
- post_ai_ask
related_guides:
- box-ai/prerequisites
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/prerequisites
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
---

# AI agent configuration versioning
Expand Down Expand Up @@ -209,4 +209,4 @@ To make sure your configurations are not affected in a negative way, you can use
```

[default-config]: g://box-ai/ai-agents/get-agent-default-config
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
16 changes: 8 additions & 8 deletions content/guides/box-ai/ai-agents/get-agent-default-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ related_endpoints:
- post_ai_text_gen
- post_ai_ask
related_guides:
- box-ai/prerequisites
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/prerequisites
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
---

# Get default AI agent configuration
Expand Down Expand Up @@ -254,7 +254,7 @@ When you set the `mode` parameter to `extract_structured` the response will be a

</Tabs>

[prereq]: g://box-ai/prerequisites
[prereq]: g://box-ai/ai-tutorials/prerequisites
[models]: g://box-ai/ai-models
[ai-agent-config]: g://box-ai/ai-agents/overrides-tutorial
[override-tutorials]: g://box-ai/ai-agents/overrides-tutorial
[ai-agent-config]: g://box-ai/ai-agents/ai-agent-overrides
[override-tutorials]: g://box-ai/ai-agents/ai-agent-overrides
4 changes: 2 additions & 2 deletions content/guides/box-ai/ai-agents/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ related_endpoints:
- post_ai_extract_structured
related_guides:
- box-ai/ai-agents/get-agent-default-config
- box-ai/ai-agents/overrides-tutorial
- box-ai/ai-agents/ai-agent-overrides
---

# AI model overrides
Expand All @@ -30,5 +30,5 @@ To see specific use cases, check the [overrides tutorial][overrides].
[ask]: e://post_ai_ask#param_ai_agent
[text-gen]: e://post_ai_text_gen#param_ai_agent
[agent-default]: g://box-ai/ai-agents/get-agent-default-config
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
[models]: g://box-ai/ai-models
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
rank: 14
related_guides:
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
- box-ai/ai-agents/get-agent-default-config
---
# AWS Claude 3.5 Sonnet
Expand Down Expand Up @@ -33,4 +33,4 @@ related_guides:
For additional information, see [official AWS Claude 3.5 Sonnet documentation][aws-claude].

[aws-claude]: https://aws.amazon.com/bedrock/claude/
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
10 changes: 5 additions & 5 deletions content/guides/box-ai/ai-models/aws-claude-3-haiku-model-card.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
rank: 15
related_guides:
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
- box-ai/ai-agents/get-agent-default-config
---
# AWS Claude 3 Haiku
Expand Down Expand Up @@ -33,4 +33,4 @@ related_guides:
For additional information, see [official AWS Claude 3 Haiku documentation][aws-claude].

[aws-claude]: https://aws.amazon.com/bedrock/claude/
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
rank: 16
related_guides:
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
- box-ai/ai-agents/get-agent-default-config
---
# AWS Claude 3 Sonnet
Expand All @@ -31,4 +31,4 @@ related_guides:
For additional information, see [official AWS Claude 3 Sonnet documentation][aws-claude].

[aws-claude]: https://aws.amazon.com/bedrock/claude/
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
rank: 17
related_guides:
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
- box-ai/ai-agents/get-agent-default-config
---
# AWS Titan Text Lite
Expand Down Expand Up @@ -32,4 +32,4 @@ although the model itself is lightweight.
For additional information, see [official AWS Titan documentation][aws-titan].

[aws-titan]: https://aws.amazon.com/bedrock/titan/
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
rank: 3
related_guides:
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
- box-ai/ai-agents/get-agent-default-config
---
# Azure OpenAI GPT-4o-2024-05-13
Expand All @@ -31,4 +31,4 @@ related_guides:
For additional information, see [official Azure OpenAI GPT-4o-2024-05-13 documentation][azure-ai-mini-4o-model].

[azure-ai-mini-4o-model]: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure#gpt-4o-and-gpt-4-turbo
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
rank: 4
related_guides:
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
- box-ai/ai-agents/get-agent-default-config
---
# Azure OpenAI GPT-4o Mini
Expand Down Expand Up @@ -48,4 +48,4 @@ Box AI API uses this model to cover the following use cases:
For additional information, see [official Azure OpenAI GPT-4o Mini documentation][azure-ai-mini-4o-model].

[azure-ai-mini-4o-model]: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure#gpt-4o-and-gpt-4-turbo
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
rank: 5
related_guides:
- box-ai/ask-questions
- box-ai/generate-text
- box-ai/extract-metadata
- box-ai/extract-metadata-structured
- box-ai/ai-tutorials/ask-questions
- box-ai/ai-tutorials/generate-text
- box-ai/ai-tutorials/extract-metadata
- box-ai/ai-tutorials/extract-metadata-structured
- box-ai/ai-agents/get-agent-default-config
---
# Azure text-embedding-ada-002
Expand All @@ -31,4 +31,4 @@ related_guides:
For additional information, see [official Azure Embeddings models documentation][azure-ai-embeddings].

[azure-ai-embeddings]: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#embeddings
[overrides]: g://box-ai/ai-agents/overrides-tutorial
[overrides]: g://box-ai/ai-agents/ai-agent-overrides
Loading
Loading