Model Configuration

ROSA supports both the OpenAI API and Azure OpenAI for its language model. Users can configure and pass either a ChatOpenAI or AzureChatOpenAI instance to the ROSA class. Here's an overview of how to set up and use these LLMs:

Using ChatOpenAI

To use the standard OpenAI API with the ChatOpenAI model:

1. Ensure you have your OpenAI API key.

2. Set up your environment variable:

   Add the following to your .env file or set it in your system environment:

   OPENAI_API_KEY=your_openai_api_key
   

3. Create a ChatOpenAI instance:

   import os
   from dotenv import load_dotenv
   from langchain_openai import ChatOpenAI
   
   load_dotenv()  # This loads the variables from .env file
   
   openai_llm = ChatOpenAI(
       model_name="gpt-4",  # or your preferred model
       temperature=0,
       max_tokens=None,
       timeout=None,
       max_retries=2,
       openai_api_key=os.getenv("OPENAI_API_KEY"),  # Using environment variable
   )
   
   # Pass the LLM to ROSA
   rosa_instance = ROSA(ros_version=2, llm=openai_llm, ...)

Using AzureChatOpenAI

To use Azure OpenAI, you'll need to create an AzureChatOpenAI instance with the appropriate configuration. There are two ways to set this up:

1. Using Azure API Management (APIM) with Tenant ID, Client ID, and Client Secret:

   Required Environment Variables:

   • APIM_SUBSCRIPTION_KEY (if required by your APIM setup)
   • AZURE_TENANT_ID
   • AZURE_CLIENT_ID
   • AZURE_CLIENT_SECRET
   • DEPLOYMENT_ID
   • API_VERSION
   • API_ENDPOINT

   Add these to your .env file or set them in your system environment.

   import os
   from dotenv import load_dotenv
   from langchain_openai import AzureChatOpenAI
   from azure.identity import ClientSecretCredential, get_bearer_token_provider
   
   load_dotenv()
   
   # Set up Azure authentication
   credential = ClientSecretCredential(
       tenant_id=os.getenv("AZURE_TENANT_ID"),
       client_id=os.getenv("AZURE_CLIENT_ID"),
       client_secret=os.getenv("AZURE_CLIENT_SECRET"),
       authority="https://login.microsoftonline.com",
   )
   
   token_provider = get_bearer_token_provider(
       credential, "https://cognitiveservices.azure.com/.default"
   )
   
   # Create AzureChatOpenAI instance
   azure_llm = AzureChatOpenAI(
       azure_deployment=os.getenv("DEPLOYMENT_ID"),
       azure_ad_token_provider=token_provider,
       openai_api_type="azure_ad",
       api_version=os.getenv("API_VERSION"),
       azure_endpoint=os.getenv("API_ENDPOINT"),
       default_headers={"Ocp-Apim-Subscription-Key": os.getenv("APIM_SUBSCRIPTION_KEY")} if os.getenv("APIM_SUBSCRIPTION_KEY") else {},
   )
   
   # Pass the LLM to ROSA
   rosa = ROSA(ros_version=2, llm=azure_llm, ...)

2. Using API Key:

   Required Environment Variables:

   • AZURE_OPENAI_API_KEY
   • DEPLOYMENT_ID
   • API_ENDPOINT

   Add these to your .env file or set them in your system environment.

   import os
   from dotenv import load_dotenv
   from langchain_openai import AzureChatOpenAI
   
   load_dotenv()
   
   # Create AzureChatOpenAI instance
   azure_llm = AzureChatOpenAI(
       azure_deployment=os.getenv("DEPLOYMENT_ID"),
       openai_api_key=os.getenv("AZURE_OPENAI_API_KEY"),
       azure_endpoint=os.getenv("API_ENDPOINT"),
   )
   
   # Pass the LLM to ROSA
   rosa = ROSA(ros_version=2, llm=azure_llm, ...)

Note: Ensure that you have the necessary environment variables set in your .env file or system environment. Always handle your API keys and secrets securely.

For more detailed information on using these models, including advanced features like tool calling, streaming, and fine-tuning, refer to the official documentation:

• Azure OpenAI Documentation
• OpenAI Documentation

Remember to handle your API keys and secrets securely, preferably using environment variables or a secure secret management system.