Azure OpenAI Insights

The 'Azure OpenAI Insights' workbook offers deep insights into Azure OpenAI usage, helping you manage costs, optimize performance, and make strategic decisions for a robust AI infrastructure.

Introduction

In the ever-evolving world of Artificial Intelligence, organizations and entities across various sectors are on a quest to leverage advanced technologies efficiently. Azure OpenAI opens a realm of possibilities, offering both challenges and excitement, particularly for those at the early stages of AI adoption.

Read more in depth in this Tech Community blog: Azure OpenAI Insights: Monitoring AI with Confidence

This workbook offers deep insights into Azure OpenAI resources and usage (Platform Metrics and Logs) and can be powerful tool in analyzing & monitoring your AI initiatives.

Structure and Views

Structure

Overview - Holistic view of Azure OpenAI resources
Monitor - Holistic view of Azure OpenAI resources Metrics
Insights - Holistic view of Azure OpenAI resources Logs

Views

Overview
- Azure OpenAI Resources by
  - SubscriptionId
  - Resource Group
  - Location
  - Kind
  - Public Network Access
  - Private Network Access
- All Azure OpenAI Resources
Monitor
- Overview
  - Requests
  - Processed Inference Tokens
  - Processed Prompt Tokens
  - Generated Completions Tokens
  - Processed FineTuned Training Hours
  - Provisioned-managed Utilization
  - Active Tokens
  - Prompt Token Cache Match Rate
  - Time to Response
- HTTP Requests
  - Requests
    - by Model Name
    - by Model Version
    - by Model Deployment Name
    - by Status Code
    - by StreamType
    - by Operation Name
    - by API Name
    - by Region
  - Time to Response
    - by Model Name
    - by Model Deployment Name
  - Prompt Token Cache Match Rate
    - by Model Name
    - by Model Deployment Name
- Token-Based Usage
  - Processed Inference Tokens
    - by Model Name
    - by Model Deployment Name
  - Processed Prompt Tokens
    - by Model Name
    - by Model Deployment Name
  - Generate Completion Tokens
    - by Model Name
    - by Model Deployment Name
  - Active Tokens
    - by Model Name
    - by Model Deployment Name
- PTU Utilization
  - Provisioned-managed Utilization
    - by Model Name
    - Model Version
    - by Model Deployment Name
    - by StreamType
    - by Region
- Fine-tuning
  - Processed FineTuned Training Hours
    - by Model Name
    - by Model Deployment Name
Insights
- Overview
  - Requests
    - by Resource
    - by Location
    - by StreamType
    - by Api Version
    - by Model Deployment Name + Operation Name
    - by Model Deployment Name
    - by Model Name + Operation Name
    - by Model Name
    - by Operation Name
    - by Avg Duration (ms)
    - by Avg Request Length (bytes)
    - by Avg Response Length (bytes)
- By CallerIP
  - Requests
  - Operation Name
  - Model Deployment Name + Operation Name
  - Model Name + Operation Name
  - Avg Duration (ms)
  - Avg Request Length (bytes)
  - Avg Response Length (bytes)
- All Logs
  - Successful requests
- Failures
  - Failed requests
    - by Resources
    - by Api Version
    - by Operation name
    - by Stream Type

Filters

Filter supported:

Model Deployment Name
Model Name
Model Version
Api Version
Operation Name
Stream Type
Location

All the filters are related to each other to allow a granular view and simplify the tracking of the logs.

Average Duration (ms)

Average Request / Response Length (bytes)

Introduction

This workbook offers deep insights into Azure OpenAI resources and usage (Platform Metrics and Logs) and can be powerful tool in analyzing & monitoring your AI initiatives.

Structure and Views

Structure

Overview - Holistic view of Azure OpenAI resources

Monitor - Holistic view of Azure OpenAI resources Metrics

Insights - Holistic view of Azure OpenAI resources Logs

Views

Overview

Azure OpenAI Resources by
- SubscriptionId
- Resource Group
- Location
- Kind
- Public Network Access
- Private Network Access
All Azure OpenAI Resources

Monitor

Overview
- Requests
- Processed Inference Tokens
- Processed Prompt Tokens
- Generated Completions Tokens
- Processed FineTuned Training Hours
- Provisioned-managed Utilization
HTTP Requests
- by Model Name
- by Model Version
- by Model Deployment Name
- by Status Code
- by StreamType
- by Operation Name
- by API Name
- by Region
Token-Based Usage
- Processed Inference Tokens
  - by Model Name
  - by Model Deployment Name
- Processed Prompt Tokens
  - by Model Name
  - by Model Deployment Name
- Generate Completion Tokens
  - by Model Name
  - by Model Deployment Name
- Active Tokens
  - by Model Name
  - by Model Deployment Name
PTU Utilization
- Provisioned-managed Utilization
  - by Model Name
  - Model Version
  - by Model Deployment Name
  - by StreamType
  - by Region
Fine-tuning
- Processed FineTuned Training Hours
  - by Model Name
  - by Model Deployment Name

Insights

Overview
- Requests
  - by Resource
  - by Location
  - by StreamType
  - by Api Version
  - by Model Deployment Name + Operation Name
  - by Model Deployment Name
  - by Model Name + Operation Name
  - by Model Name
  - by Operation Name
  - by Avg Duration (ms)
  - by Avg Request Length (bytes)
  - by Avg Response Length (bytes)
By CallerIP
- Requests
- Operation Name
- Model Deployment Name + Operation Name
- Model Name + Operation Name
- Avg Duration (ms)
- Avg Request Length (bytes)
- Avg Response Length (bytes)
All Logs
- Successful requests
Failures
- Failed requests
  - by Resources
  - by Api Version
  - by Operation name
  - by Stream Type

Filters

Filter supported:

Model Deployment Name

Model Name

Model Version

Api Version

Operation Name

Stream Type

Location

All the filters are related to each other to allow a granular view and simplify the tracking of the logs.

Average Duration (ms)

Average Request / Response Length (bytes)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Azure OpenAI Insights

Introduction

Structure and Views

Structure

Views

Filters

Azure OpenAI Insights

Introduction

Structure and Views

Structure

Views

Filters

Releases: dolevshor/Azure-OpenAI-Insights