Skip to content

Latest commit

 

History

History
382 lines (259 loc) · 23.2 KB

README.md

File metadata and controls

382 lines (259 loc) · 23.2 KB
Raw

Contoso Chat Retail with .NET and Semantic Kernel

This sample demonstrates how to build, evaluate, and deploy, a retail copilot application end-to-end using Azure AI and Cosmos DB. It's built using .NET with the Semantic Kernel library and Prompty prompt files.

Table of Contents

Open in GitHub Codespaces Open in Dev Containers

What is this sample?

In this sample, we present Contoso Outdoors, a conceptual store specializing in outdoor gear for hiking and camping enthusiasts. This virtual store enhances customer engagement and sales support through an intelligent chat agent. Artificial Intelligence is used to integrate into the customer service experience, offering responses that are not only relevant but also personalized, drawing from an extensive product catalog and individual customer purchase histories.

The sample uses the following Azure technologies:

  • Azure AI Search to create and manage search indexes for product catalog data.
  • Azure Cosmos DB to store and manage customer purchase history data.
  • Azure OpenAI to deploy and manage key models for our copilot workflow.
    • text-embeddings-ada-002 for vectorizing user queries.
    • gpt-35-turbo for generating chat responses.

It exposes a REST endpoint that can be called to interact with the chat agent and is implemented using:

  • Semantic Kernel to access AI models, work with data stores, integrate prompts, and evaluate prompt/LLM performance.
  • Prompty to simplify prompt creation & iteration for this copilot scenario.
  • ASP.NET Core WebAPI to expose REST endpoints.

By exploring and deploying this sample, you will learn to:

  1. Build a retail copilot application using the RAG pattern.
  2. Ideate & iterate on application using Semantic Kernel and Prompty.
  3. Build & manage the solution using the Azure AI platform & tools.
  4. Provision & deploy the solution using the Azure Developer CLI.
  5. Support Responsible AI practices with evaluation & content safety.

Features

The project comes with:

  • Sample model configurations, chat and evaluation prompts for a RAG-based copilot app.
  • Prompty assets to simplify prompt creation & iteration for this copilot scenario.
  • Sample product and customer data for retail application scenario
  • Sample application code for copilot chat and evaluation workflows.
  • Sample azd-template configuration for managing application on Azure
  • Managed Identity configuration as a best practice for managing sensitive credentials.

The sample is also a signature application for demonstrating the new capabilities of the Azure AI platform. Expect regular updates to showcase cutting-edge features and best practices for generative AI development.

Architecture Diagram

The Contoso Chat application implements a retrieval augmented generation pattern to ground the model responses in your data. The architecture diagram below illustrates the key components and services used for implementation and highlights the use of Azure Managed Identity to reduce developer complexity in managing sensitive credentials.

Architecture Diagram

Getting Started

Pre-requisites

You will also need to validate the following requirements:

  • Access to semantic ranker feature for your search service tier and deployment region.
  • Access to sufficient Azure OpenAI quota for your selected models and deployment region.
  • Ability to provision Azure Monitor (Free tier)
  • Ability to deploy to Azure Container Apps (Free tier)

Applications to install for local development (Optional)

  • Visual Studio Code - Download it for free here.
  • Install azd
    • Windows: winget install microsoft.azd
    • Linux: curl -fsSL https://aka.ms/install-azd.sh | bash
    • MacOS: brew tap azure/azd && brew install azd

Deploy

Step 1: Development Environment

The repository is instrumented with a devcontainer.json configuration that can provide you with a pre-built environment that can be launched locally, or in the cloud. You can also elect to do a manual environment setup locally, if desired. Here are the three options in increasing order of complexity and effort on your part.

Pick one!

  • Pre-built environment, in cloud with GitHub Codespaces, described in the Quickstart.
  • Pre-built environment, on device with Docker Desktop
  • Manually setup environment, on device with .NET and NuGet installed

The first approach is recommended for minimal user effort in startup and maintenance. The third approach will require you to manually update or maintain your local environment, to reflect any future updates to the repo.

1.1: Quickstart with Codespaces

The recommended option!

  1. Click the green Code button on this repository.

  2. Select the Codespaces tab and click Create codespace...

    • You can also click this button: Open in GitHub Codespaces

  3. This should open a new browser tab with a Codespaces container setup process running.

  4. On completion, this will launch a Visual Studio Code editor in the browser, with all relevant dependencies already installed in the running development container beneath.

Congratulations! Your cloud dev environment is ready!

Once you've launched Codespaces you can proceed to Deploy in Step 2.

1.2: Pre-Built Environment, in cloud (Docker)

This option uses the same devcontainer.json configuration, but launches the development container in your local device using Docker Desktop.

To use this approach, you need to have the following tools pre-installed in your local device:

  • Visual Studio Code (with Dev Containers Extension)
  • Docker Desktop (community or free version is fine)

Make sure your Docker Desktop daemon is running on your local device. After this:

  • Fork this repo to your personal profile
  • Clone that fork to your local device
  • Open the cloned repo using Visual Studio Code
  • If your Dev Containers extension is installed correctly, you will be prompted to "re-open the project in a container" - just confirm to launch the container locally.
  • Alternatively, you may need to trigger this step manually. See the Dev Containers Extension for more information.

Once your project launches in the local Docker desktop container, you should see the Visual Studio Code editor reflect that connection in the status bar (blue icon, bottom left).

Congratulations! Your local dev environment is ready!

Once you've launched your docker container environment you can proceed to step 2.

Step 1.3: Manually Setup Environment on device

In order to run this sample locally you will need to:

  1. Clone this sample locally:

    git clone https://github.com/Azure-Samples/chat-rag-openai-csharp-prompty.git

  2. Open the repo in VS Code and navigate to the src directory

    cd 
code .
cd ./src/ContosoChatAPI

  3. Install the .NET SDK

  4. Install the newest version of PowerShell

    • Open PowerShell and use winget install --id Microsoft.Powershell --source winget

  5. Install the C# Dev Kit in VS Code

    • Open the VS Code Extensions tab
    • Search for "C# Dev Kit"
    • Install the extension

  6. Install the Azure CLI for your device OS

2. Create Azure resources

We setup our development ennvironment in the previous step. In this step, we'll provision Azure resources for our project so they're ready to use for developing our LLM Application.

2.1 Authenticate with Azure

Start by connecting your Visual Studio Code environment to your Azure account:

  1. Open the terminal in VS Code and use command az login.
  2. Complete the authentication flow.

If you are running within Codespaces, use these instructions to login instead:

  1. Open the terminal in VS Code and use command az login --use-device-code
  2. The console message will give you an alphanumeric code
  3. Navigate to https://microsoft.com/devicelogin in a new tab
  4. Enter the code from step 2 and complete the flow.

In either case, verify that the console shows a message indicating a successful authentication. Set your subscription in your enviroment with:

az account set -s <SUB_ID>

Congratulations! Your VS Code session is now connected to your Azure subscription!

2.2 Provision with Azure Developer CLI

For this project, we need to provision multiple Azure resources in a specific order; we'll use the Azure Developer CLI (or azd) to do this, following the steps below. Visit the azd reference for more details on tool syntax, commands and options.

2.2.1 Install azd

  • If you setup your development environment manually, follow these instructions to install azd for your local device OS.
  • If you used a pre-built dev container environment (e.g., GitHub Codespaces or Docker Desktop) the tool is pre-installed for you.
  • Verify that the tool is installed by typing azd version in a terminal.

2.2.2 Authenticate with Azure

  • Start the authentication flow from a terminal: 
    azd auth login
  • This should activate a Device Code authentication flow as shown below. Just follow the instructions and complete the auth flow till you get the Logged in on Azure message indicating success. 
    Start by copying the next code: <code-here>
Then press enter and continue to log in from your browser...

2.2.3 Provision and Deploy

  • Run the following command to provision all resources. Note: This will take awhile to complete, usually around 10-15 minutes.

    azd up

  • During provisioning, you will be prompted for the Azure subscription you want you use and the region. You will be prompted for two different region selections: the first one is the primary location for all resources, the second is a location field specifically for where the OpenAI resource should be created.

    • At this time, we recommend using Sweden Central or East US 2 regions for OpenAI resources.
    • For Azure AI Search we recommend using the East US region, due to limited regional availability currently.

  • On completion, it automatically invokes a postprovision.sh script, that will be located in the infra/hooks folder. This will attempt to log you into Azure. You may see something like this. Just follow the provided instructions to complete the authentication flow.

    No Azure user signed in. Please login.

  • Once logged in, the script will do the following for you:

    • Create the .azure/<env>/config.json in the local device
    • Populate .azure/<env>/.env with required environment variables

That's it! You should now be ready to continue; start by completing the verification steps below and taking any troubleshooting actions identified.

2.2.4 Verify Provisioning

The script will have set up a dedicated resource group with the required resources described earlier in the architecture section, including:

  • Cosmos DB
  • Azure OpenAI
  • Search service (Azure AI Search) resource\
  • An Azure Container Apps (ACA) resource hosting the REST endpoint

The script will set up an Azure AI Studio project with the following model deployments created by default, in a relevant region that supports them. Your Azure subscription must be enabled for Azure OpenAI access.

  • gpt-3.5-turbo
  • text-embeddings-ada-002

The Azure AI Search resource will have Semantic Ranker enabled for this project, which requires the use of a paid tier of that service. It may also be created in a different region, based on availability of that feature.

2.3 Configure endpoints in the .NET Application

The project now needs to have access to Azure, to do this, open the .\src\ContosoChatAPI\ContosoChatAPI\appsettings.json file in your project and change the following variables. You can use the \.azure\<env-name>\.env file, generated during Azure resource provisioning, to find the correct configuration values to use.

.env variable appsettings.json value
APPINSIGHTS_CONNECTIONSTRING ApplicationInsights -> ConnectionString
AZURE_OPENAI_ENDPOINT OpenAi -> Endpoint
CosmosDb__Endpoint CosmosDb -> Endpoint
AzureAISearch__Endpoint AzureAISearch -> Endpoint

3. Running the application

In Visual Studio Code, start debugging (e.g. by pressing the F5 key) - when prompted, select C# as your language, and then choose the ContosoChatAPI project.

If you're using GitHub Codespaces, click to confirm when you're prompted to open a URL in the browser. Once it's open, you'll see an error - change the URL by adding "/swagger/" to the end of it and load the page.

If you're not using GitHub Codespaces, the browser will open to the correct URL.

3.1 Using the aplication with Swagger

Your browser should now be open to the Swagger UI (used for documenting and testing the Web API):

Swagger Web Page

Click on the /ContosoChat API, then click the Try it out button and provide the following parameters:

- customerId: `3`
- question: `What can you tell me about jackets?`

Click the Execute button to send the question to the chat agent.

  • An HTTP POST request will then be sent to the backend Web API.
  • Semantic Kernel will then be used to query for additional context data from CosmosDB.
  • A prompt, defined using the Prompty format, will be created combining the customerId, question, and context data.
  • This prompt will then be sent to the Azure OpenAI service which will send back an answer to the chat.

A response will then be shown in the Swagger UI, with content similar to:

Answer: Sure, Michael! Let me tell you about our jackets. One of our popular jackets is the Summit Breeze Jacket (catalog id 23). It's a lightweight and windproof jacket with a water-resistant fabric. It's perfect for outdoor adventures and comes in a sleek black color. 🌬️🧥

If you're interested in jackets, you might also like the TrailWalker Hiking Shoes (catalog id 5) for a complete outdoor outfit. These shoes are rugged, waterproof, and provide excellent grip. They will keep your feet comfortable and protected during your hikes. 👟🏔️

Let me know if you need any more information!

4. Calling the API deployed in Azure

During Azure resource provisioning, an Azure Container App was deployed with the REST endpoint we just tested locally using Swagger. To call and test the deployed endpoint directly:

  • Get the endpoint URL from the .env file that was created during provisioning.
    • Open the src\ContosoChatAPI\ContosoChatAPI\appsettings.json file in your project.
    • Copy the value of the SERVICE_ACA_URI variable.
  • POST to the api endpoint with the customerId and question, using a tool like curl:
curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"customerId":"3","question":"Tell me about your jackets?"}' \
  http://<service-aca-uri>/ContosoChat

Costs

You can estimate the cost of this project's architecture with Azure's pricing calculator

  • Azure OpenAI - Standard tier, GPT-35-turbo and Ada models. See Pricing
  • Azure AI Search - Basic tier, Semantic Ranker enabled See Pricing
  • Azure Cosmos DB for NoSQL - Serverless, Free Tier See Pricing
  • Azure Monitor - Serverless, Free Tier See Pricing
  • Azure Container Apps - Severless, Free Tier See Pricing

Security Guidelines

This template uses Managed Identity or Key Vault to eliminate the need for developers to manage credentials. Applications can use managed identities to obtain Microsoft Entra tokens without having to manage any credentials.

Additionally, we have added a GitHub Action tool that scans the infrastructure-as-code files and generates a report containing any detected issues.

To ensure best security practices in your repo, we recommend anyone creating solutions based on our templates ensure that the Github secret scanning setting is enabled in your repos.

Resources

Troubleshooting

Have questions or issues to report? Please open a new issue after first verifying that the same question or issue has not already been reported. If your issue has already been reported, please add any additional comments you may have, to the existing issue.

Verifying permissions

Depending on your environment, you may see permissions errors during the provisioning of Azure resources. Here are some common permissions you will need enabled for your account:

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct.

For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines.

Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.

Any use of third-party trademarks or logos are subject to those third-party's policies.