Internal Server Error and other issues

[privileger]

Hi everyone,

I deployed this repo a couple of months ago and used it without any issues. Today, I deployed it again in a demo environment, but the front-end application is not running properly, showing an 'Internal Server Error.' The backend application is running; however, after uploading the document for indexing, it is throwing multiple errors.

The template was deployed without modifications. I only added the Japanese language "ja-JP" in the list of supported languages during the deployment.

Please advise how to resolve this issue.

[Roopan-Microsoft]

Hey @privileger, As mentioned in the issue. Please try to redeploy as this fix has been merged as part of #1305.

Please raise an issue in case you face any blocker or issue.

Happy to help!

[privileger]

@Roopan-Microsoft
Thanks for the update. I just re-deployed this repo, and the reported issue is gone. However, the front end is still acting strange because I am receiving an error: "An error occurred. Please try again. If the problem persists, please contact the site administrator." After clicking the OK button, the LLM no longer returns answers based on the RAG's content. Please advise.

[Roopan-Microsoft]

Hey @privileger, It looks strange.

Could you please investigate further using Application Insights to check for any exceptions or additional details? I believe there might be relevant information that could help us diagnose the issue more precisely.

Also please let us know if any configuration change happend, so that we can try to repro the same from our end.

Looking forward to your findings.

[privileger]

@Roopan-Microsoft
Here you go.

[privileger]

@Roopan-Microsoft

Some additional screenshots.

[Prasanjeet-Microsoft]

@privileger In the last screenshot you provided, could you please click on the 500 response code and provide additional details? I have provided an example below for your reference.

Click on 500 response code:

Click on suggested view details:

Click on EXCEPTION:

[privileger]

@Roopan-Microsoft Here you go. Hope it helps.

[Roopan-Microsoft]

Hey @privileger - Thanks for more details, we are looking into this. Will keep you posted.

[Roopan-Microsoft]

Hey @privileger,

We see multiple issues, Let me explain one by one,

Issue 1: ( Open AI Content Management Policy)
This issue is due to the question which may be harmful content or anything sensitive. Open AI will filter those and will respond as 400. Which is seems to be expected. (App should work for normal queries)
Let us know the question which you asked. we will verify from our end.

Issue 2: (No Index found)
Based on the exceptions, we see there is missing index. Please check in your Search Service resource.
Serach Service -> Index (Check if you have any index called conversations)

Please let us know your observations.

Also please let us know if your issue exists still or on specific scenarios.

Happy to Help!

[Prasanjeet-Microsoft]

@privileger Could you please provide us with an update on the above comments?

FYI @Roopan-Microsoft

[privileger]

@Roopan-Microsoft Hello, and thank you for the follow-up. I deployed a solution today in the Japan East region, and it worked for a while before the front end generated the same error. Please refer to the screenshots below. I’ve observed that the chatbot's functionality seems to depend on the context.

During deployment, I only modified one parameter, removing all languages except en-US and adding ja-JP.

I then uploaded a 129-page Word document written in Japanese and attempted to query the LLM based on its contents. The document included text, tables, and images. While it worked for some prompts, it eventually resulted in the same error as before.

Please refer to the screenshot below.

[privileger]

@Roopan-Microsoft
Please find below some additional screenshots.

[Prasanjeet-Microsoft]

Hello @privileger , After reviewing the screenshots, it looks like you are experiencing two types of issues: one related to content management policy and the other concerning authorization for machine learning services.

1st Issue: Content Management Policy

• As mentioned in the previous comment, this issue arises from the content filtering system, which identifies and takes action on certain categories of potentially harmful content in both input prompts and output completions. This behavior appears to be expected.

• Therefore, we recommend asking different questions or

• Try to deploy this accelerator using the gpt-4o (2024-05-13) GlobalStandard model, which offers superior performance in non-English languages, if you intend to deploy only in the Japan East region and ask questions in japanese language.

• Alternatively, you could also deploy this accelerator in a different region using the gpt-4o (2024-05-13) model and gpt-4o (2024-08-06) model to achieve better performance in non-English languages.

Please find a few links below for your reference:
https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/models#model-summary-table-and-region-availability
https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#global-standard-model-availability
https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models#fine-tuning-models

2nd Issue: Authorization for Machine Learning Services

• It seems the service principal that you are using doesn't have rights within that tenant.

• Tenants have subscriptions and service principals belong to tenants. Azure resource manager also exposes role based authorization for a given principal, which would give it rights on Azure resources. It appears the service principal doesn't have rights to read from that subscription.

• Go to portal and find your subscription, click on Access Control (IAM) and then click on Add role assignment with correspond service principal which you use to acquire token.

• After you have given successful permission, refresh and try again.

Note: If you dont have service principal then you can follow below steps

To check access in an Azure subscription and assign the Reader or Contributor role, follow these steps:

Step 1: Sign in to the Azure Portal

1. Go to the Azure Portal.
2. Sign in with your Azure account credentials.

Step 2: Check Access in the Subscription

1. In the left-hand navigation pane, select Subscriptions.
2. Choose the subscription you want to check.
3. In the subscription menu, click on Access control (IAM).
4. On the Check access tab, click the View my access button.
5. An assignments pane appears that lists your access at this scope and inherited to this scope. If you don't have reader or contributor role then follow further steps.

Step 3: Assign the Reader or Contributor Role

1. In the Access control (IAM) section, click on the + Add button.
2. Select Add role assignment.
3. On the Role tab, select a role that you want to use. You can search for a role by name or by description. (Select either Reader or Contributor, depending on the level of access you want to assign)
4. Click Next
5. On the Members tab, select User, group, or service principal to assign the selected role to one or more Microsoft Entra users, groups, or service principals (applications).
6. Click Select members.
7. Find and select the users, groups, or service principals. (You can type in the Select box to search the directory for display name or email address.)
8. Use the Select option to find and select the user (your name) or group you want to assign the role to.
9. Click Review + assign to assign the role
10. After a few moments, the security principal is assigned the role at the selected scope

Step 4: Confirm the Assignment

1. After saving, you should see the new role assignment listed in the role assignments section.

Notes

• Make sure you have the necessary permissions to assign roles in the subscription (typically, you need to be an Owner or have the User Access Administrator role).
• Your updates might take a while to be processed and reflected in the portal.

Please find below links for your reference:
https://learn.microsoft.com/en-us/azure/role-based-access-control/check-access#step-2-check-access-for-a-user
https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal
https://learn.microsoft.com/en-us/azure/role-based-access-control/quickstart-assign-role-user-portal

[Prasanjeet-Microsoft]

@privileger Could you please let us know if you’re able to proceed ahead without any issues? If so, can we close this issue?

[Prasanjeet-Microsoft]

Could you please let us know if you’re able to proceed ahead without any issues? If so, can we close this issue?

Hello @privileger, just checking in to see if you’re able to proceed without any issues. If everything looks good, can we go ahead and close this issue?

Thank you!

[privileger]

@Prasanjeet-Microsoft I need a bit more time to redeploy the solution and check everything.

All Incan say for now that the document I fed to Azure Open AI is related to the health and safety, so I am not sure why the content was blocked by content management policy. Can I get more details in this?

Also, what parameters should be changed in order to activate 4o model?

[Prasanjeet-Microsoft]

@privileger Do you prefer to deploy in the Japan East region or is any region acceptable?

[privileger]

@Prasanjeet-Microsoft For the testing purposes, it can be deployed in any region, however, the production environment must be deployed in Japan East.

[Prasanjeet-Microsoft]

Hello @privileger, If you want to activate the gpt-4o model, you need to update the default values of the parameters to the new values listed below during the new deployment:

Azure Open AI Model : gpt-4o
Azure Open AI Model Name : gpt-4o
Azure Open AI Model Version : 2024-05-13

Please find screenshot for your reference:

Note: Please refer below link for Model summary table and region availability before doing the deployment with gpt-4o model

https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/models#model-summary-table-and-region-availability

[privileger]

@Prasanjeet-Microsoft Hello. Thank you for resolving this issue. I will close a ticket for now.