Skip to content

Commit

Permalink
update RAG alert and RAG link
Browse files Browse the repository at this point in the history
  • Loading branch information
akiliu00512 committed Oct 31, 2024
1 parent 15db197 commit 1faa1d8
Show file tree
Hide file tree
Showing 9 changed files with 502 additions and 119 deletions.
50 changes: 28 additions & 22 deletions docs/content/patterns/artificial intelligence/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,31 +7,37 @@ geekdocCollapseSection: true

There are numerous ways to implement AI solution on Azure, and each comes with its own monitoring solution. Monitoring AI solutions involves a combination of the infra or paas resources, along with monitoring any utilization metrics that can be exposed through the platform or other tooling. This page will summarize the recommended monitoring solutions for different scenarios.

### AI on Azure platforms (PaaS)

Common AI Ready infrastructure on Azure may include services including Azure AI Hub, Azure AI Services including Azure OpenAI, AI Search etc. Specific workloads like Azure Kubernetes Services, API Management, App Services are also used to build enterprise level AI applications. Below table lists quick links to alert guidelines to most commonly used services, for other Azure service in your architecture, refer to the [Azure Resource](../../../../azure-monitor-baseline-alerts/services/) which provides more comprehensive lists.

| Services | Resource Type |
| --- | --- |
| Azure AI Studio Hub/Azure Machine Learning Hub | [Microsoft.MachineLearningServices/workspaces](../../../../azure-monitor-baseline-alerts/services/MachineLearningServices/workspaces/) |
| Azure AI Search | [Microsoft.Search/searchServices](../../../../azure-monitor-baseline-alerts/services/Search/searchServices/) |
| Azure AI Services | [Microsoft.CognitiveServices/accounts](../../../../azure-monitor-baseline-alerts/services/CognitiveServices/accounts) |
| Azure Kubernetes Services | [Microsoft.ContainerService/managedClusters](../../../../azure-monitor-baseline-alerts/services/ContainerService/managedClusters/) |
| Azure App Services | [Microsoft.Web/sites](../../../../azure-monitor-baseline-alerts/services/Web/sites/) |
| Azure API Management | [lMicrosoft.ApiManagement/service](../../../../azure-monitor-baseline-alerts/services/ApiManagement/service/) |
| Azure Container Apps | [Microsoft.App/containerApps](../../../../azure-monitor-baseline-alerts/services/App/containerApps//) |
| Azure Functions Apps | [Microsoft.Web/sites](../../../../azure-monitor-baseline-alerts/services/Web/sites/) |
| Azure Cosmos DB | [Microsoft.DocumentDB/databaseAccounts](../../../../azure-monitor-baseline-alerts/services/DocumentDB/databaseAccounts/) |
| Azure SQL Database - managedInstances | [Microsoft.Sql/managedInstances](../../../../azure-monitor-baseline-alerts/services/Sql/managedInstances/) |
| Azure SQL Database - server | [Microsoft.Sql/servers/databases](../../../../azure-monitor-baseline-alerts/services/Sql/servers/) |
| Azure Database for MySQL | [Microsoft.DBforMySQL/servers](../../../../azure-monitor-baseline-alerts/services/DBforMySQL/servers/) |
| Azure Database for PostgreSQL | [Microsoft.DBforPostgreSQL/servers](../../../../azure-monitor-baseline-alerts/services/DBforPostgreSQL/servers//) |

### AI on Azure infrastructure (IaaS)
### AI on Azure Platforms (PaaS)

Common AI Ready infrastructures on Azure may contain services such as Azure AI Hub, Azure AI Services (including Azure OpenAI) and AI Search. Specific workloads like Azure Kubernetes services, API Management and App Services are also frequently used to build enterprise-level AI applications.
The table below provides quick links to alert guidelines for the most commonly used services. For other Azure services in your architecture, please refer to the [Azure Resource](../../services/), which offers comprehensive lists.

|Services|Resource Type|
|--- |--- |
|Azure AI Studio Hub/Azure Machine Learning |[Microsoft.MachineLearningServices/workspaces](../../services/machineLearningServices/workspaces/)|
|Azure AI Search|[Microsoft.Search/searchServices](../../services/Search/searchServices/)|
|Azure AI Services |[Microsoft.CognitiveServices/accounts](../../services/CognitiveServices/accounts/)|
|Azure Kubernetes services |[Microsoft.ContainerService/managedClusters](../../services/ContainerService/managedClusters/)|
|Azure App Services |[Microsoft.Web/sites](../../services/Web/sites/)|
|Azure API Management |[Microsoft.ApiManagement/service](../../services/ApiManagement/service/)|
|Azure Container Apps |[Microsoft.App/containerApps](../../services/App/containerApps/)|
|Azure Functions Apps |[Microsoft.Web/sites](../../services/Web/sites/)|
|Azure Cosmos DB |[Microsoft.DocumentDB/databaseAccounts](../../services/DocumentDB/databaseAccounts/)|
|Azure SQL Database - managedInstances |[Microsoft.Sql/managedInstances](../../services/Sql/managedInstances/)|
|Azure SQL Database - server |[Microsoft.Sql/servers/databases](../../services/Sql/servers/)|
|Azure Database for MySQL - flexibleServers|[Microsoft.DBforMySQL/flexibleServers](../../services/DBforMySQL/flexibleServers/)|
|Azure Database for MySQL - servers |[Microsoft.DBforMySQL/servers](../../services/DBforMySQL/servers/)|
|Azure Database for PostgreSQL - flexibleServers|[Microsoft.DBforPostgreSQL/flexibleServers](../../services/DBforPostgreSQL/flexibleServers/)|
|Azure Database for PostgreSQL - servers|[Microsoft.DBforPostgreSQL/servers](../../services/DBforPostgreSQL/servers/)|



### AI on Infrastructure (IaaS)

Running AI workloads on Azure infrastructure involves monitoring each of the components of the solution, including virtual machines, storage, and networking. Refer to the defined metrics in [HPC](../../specialized/hpc/Alerting-and-Monitoring.md). For monitoring the GPU/CPU metrics, use [Moneo](https://github.com/Azure/Moneo)



### AI Specialized Workload Patterns

#### GPT-RAG (coming soon)
#### [GPT-RAG](./rag/)
20 changes: 20 additions & 0 deletions docs/content/patterns/artificial intelligence/rag/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: GPT-RAG
geekdocCollapseSection: true
---

## Overview

The Retrieval-Augmented Generation (RAG) pattern is an industry-standard approach to building applications that use large language models to reason over specific or proprietary data that is not already known to the large language model.

This page provides the alert settings for AI RAG pattern setup. It contains relevant metrics and threshold recommendations for key services involved in a RAG pattern architecture.For a reference architecture design of RAG, see [GPT-RAG](https://github.com/Azure/GPT-RAG).

Below is a basic architecture of RAG implementation

![](https://github.com/Azure/GPT-RAG/blob/main/media/architecture-GPT-RAG-Basic.png?raw=true)

We may update these settings as we continue to work with a breadth of customers.

## Alerts

{{< ragMetricAlerts >}}
95 changes: 95 additions & 0 deletions docs/layouts/shortcodes/ragMetricAlerts.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
<div><table>
<tr>
<th>Alert Name</th>
<th>Component</th>
<th>Metric</th>
<th>Aggregation</th>
<th>Operator</th>
<th>Threshold</th>
<th>Window</th>
<th>Frequency</th>
<th>Severity</th>
<th>Scope</th>
<th>Support for Multiple Resources</th>
<th>Verified</th>
<th>References</th>
</tr>

{{ range $category, $types := $.Site.Data }}
{{ range $type, $rules := $types }}
{{ range $rules.alerts }}
{{ if or (eq .visible true) (eq $.Site.Params.ambaDevMode true) }}
{{ if and (eq .type "Metric") (in .tags "rag") }}
{{ $data := newScratch }}
{{ if isset . "deployments" }}
{{ range where .deployments "type" "Policy" }}
{{ if and (in .tags "rag") }}
{{ $data.Set "name" .name }}
{{ $data.Set "url" (relURL (path.Join "services" $category $type .template)) }}
{{ $data.Set "scope" .properties.scope }}
{{ $data.Set "multiResource" .properties.multiResource }}
{{ end }}
{{ end }}
{{ end }}
<tr>
<td>
<a href='{{ $data.Get "url" }}'>{{ $data.Get "name" }}</a>
</td>
<td>
{{ .properties.metricNamespace }}
</td>
<td>
{{ .properties.metricName }}
</td>
<td>
{{ .properties.timeAggregation }}
</td>
<td>
{{ .properties.operator }}
</td>
<td>
{{ if eq .properties.criterionType "DynamicThresholdCriterion" }}
dynamic
{{ else }}
{{ .properties.threshold }}
{{ end }}
</td>
<td>
{{ .properties.windowSize }}
</td>
<td>
{{ .properties.evaluationFrequency }}
</td>
<td>
{{ .properties.severity }}
</td>
<td>
{{ $data.Get "scope" }}
</td>
<td>
{{ if ($data.Get "multiResource") }}
Yes
{{ else }}
No
{{ end }}
</td>
<td>
{{ if .verified }}
Y
{{ else }}
N
{{ end }}
</td>
<td>
{{ range .references }}
<a href="{{ .url }}" target="_blank">{{ .name }}</a>
{{ end }}
</td>
</tr>
{{ end }}
{{ end }}
{{ end }}
{{ end }}
{{ end }}

</table></div>
Loading

0 comments on commit 1faa1d8

Please sign in to comment.