Skip to content

Commit

Permalink
Merge branch 'vNext-Dev' of https://github.com/microsoft/PubSec-Info-…
Browse files Browse the repository at this point in the history
…Assistant into geearl/6323-large-tables
  • Loading branch information
georearl committed Jan 17, 2024
2 parents 3b3acf3 + 3faba43 commit 0f8b348
Show file tree
Hide file tree
Showing 4 changed files with 81 additions and 0 deletions.
4 changes: 4 additions & 0 deletions docs/costestimator.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@ The Azure pricing calculator helps estimate costs by considering the amount of d
---
### Azure Services

<<<<<<< HEAD
The following list of Azure Services will be deployed for IA Accelerator, version 1.0:
=======
The following list of Azure Services will be deployed for IA Accelerator, version 0.4 delta:
>>>>>>> 3faba43c904db668697c16df33a11c67629bd490
- App Service [:link:](https://azure.microsoft.com/en-ca/pricing/details/app-service/linux/)
- Azure Function(App Service plan) [:link:](https://azure.microsoft.com/en-ca/pricing/details/functions/#pricing)
Expand Down
8 changes: 8 additions & 0 deletions docs/deployment/autoscale_sku.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,22 +105,30 @@ More information can be found [here.](https://azure.microsoft.com/en-us/pricing/

### Enrichment Message Dequeue Parameter
<<<<<<< HEAD
<<<<<<< HEAD
There exist a property that can be set in the local.env file called `DEQUEUE_MESSAGE_BATCH_SIZE` and is defaulted in the `infra/main.bicep` and `app/enrichment/app.py` to the value of **3**. This means the app will process 3 messages from the queue at a time. This is found to be the most opitmal with the existing configuration but can be increased if you also increase the enrichment app service SKU. It is important to note that there will be issues if it is increased more than the app service SKU can handle.
=======
There exist a property that can be set in the local.env file called `DEQUEUE_MESSAGE_BATCH_SIZE` and is defaulted in the `infra/main.bicep` and `app/enrichment/app.py` to the value of **3**. This means the app will process 3 messages from the queue at a time. This is found to be the most optimal with the existing configuration but can be increased if you also increase the enrichment app service SKU. It is important to note that there will be issues if it is increased more than the app service SKU can handle.
>>>>>>> c3dca962f39fa8834aa70895953aef3409b92870
=======
There exist a property that can be set in the local.env file called `DEQUEUE_MESSAGE_BATCH_SIZE` and is defaulted in the `infra/main.bicep` and `app/enrichment/app.py` to the value of **3**. This means the app will process 3 messages from the queue at a time. This is found to be the most optimal with the existing configuration but can be increased if you also increase the enrichment app service SKU. It is important to note that there will be issues if it is increased more than the app service SKU can handle.
>>>>>>> 3faba43c904db668697c16df33a11c67629bd490
### Customization

To customize the App Service Plans SKU settings, modify the `sku` parameters in the specified Bicep file and run the `make deploy` or `make infrastructure`command.

This can also be adjusted in the Azure Portal.

<<<<<<< HEAD
<<<<<<< HEAD
**Note:** Adjusting the scale or Tier can cause outages until the redeployment occurrs.
=======
**Note:** Adjusting the scale or Tier can cause outages until the redeployment occurs.
>>>>>>> c3dca962f39fa8834aa70895953aef3409b92870
=======
**Note:** Adjusting the scale or Tier can cause outages until the redeployment occurs.
>>>>>>> 3faba43c904db668697c16df33a11c67629bd490

### Steps to Scale Up:
Expand Down
69 changes: 69 additions & 0 deletions docs/knownissues.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ Error: This subscription cannot create CognitiveServices until you agree to Resp
***IMPORTANT:*** In some instances, an older subscription that has already had the "Responsible AI Notice" accepted for the old "Cognitive Services multi-service account" may require this process to be repeated for the new "Azure AI Service".
<<<<<<< HEAD
=======
***IMPORTANT:*** In some instances, an older subscription that has already had the "Responsible AI Notice" accepted for the old "Cognitive Services multi-service account" may require this process to be repeated for the new "Azure AI Service".
>>>>>>> 3faba43c904db668697c16df33a11c67629bd490
---
## Error "Your administrator has configured the application infoasst_web_access_xxxxx to block users..."
Expand Down Expand Up @@ -127,4 +132,68 @@ make: *** [Makefile:18: infrastructure] Error 1
### Solution
<<<<<<< HEAD
=======
The error is interesting as it seems to indicate the media service cannot be created. This is not the case, it does work in regions where VI and Media Services are available. I have updated this to an enhancement and we will add a ticket to the board to action this when VI can be deployed without this supporting service.
## Error: Token limit often exceeded with PDF files
### Solution:
The root of this is table processing. If a table is greater than our target token count for a chunk, this is not respected.Essentially tables are not chunked, but treated as units. We have added a task to our board to split tables by chunk size and repeat the table header rows in each chunk..
When we switched to using unstructured.io for non-PDF documents, we were aware of the same issue there. They were planning on adding this feature. So, we need to make the change in our code, and follow up with unstructured to confirm if this has been fixed and update that path also.
This issue has been updated to an enhancement.
## Error:
If you encounter an error similar to the one below that indicates your device must be managed.
```
ERROR: AADSTS530003: Your device is required to be managed to access this resource.
Trace ID: xxxxxxxx-xxxx-xxxx-xxxxxxxx
Correlation ID: xxxxxxxx-xxxx-xxxx-xxxxxxxx
Timestamp: 2023-10-05 19:54:05Z
Interactive authentication is needed. Please run:
az login --scope https://graph.microsoft.com//.default
make: *** [Makefile:18: infrastructure] Error 1
```
### Solution
You will need to open your Codespace in VSCode on your managed device. Please read more about opening your [CodeSpace using VSCode](/docs/deployment/developing_in_a_codespaces.md#using-github-codespaces-in-visual-studio-code).
## Error:
If you encounter an error similar to the one below that indicates your device must be managed.
```
ERROR: AADSTS530003: Your device is required to be managed to access this resource.
Trace ID: xxxxxxxx-xxxx-xxxx-xxxxxxxx
Correlation ID: xxxxxxxx-xxxx-xxxx-xxxxxxxx
Timestamp: 2023-10-05 19:54:05Z
Interactive authentication is needed. Please run:
az login --scope https://graph.microsoft.com//.default
make: *** [Makefile:18: infrastructure] Error 1
```
### Solution
You will need to open your Codespace in VSCode on your managed device. Please read more about opening your [CodeSpace using VSCode](/docs/deployment/developing_in_a_codespaces.md#using-github-codespaces-in-visual-studio-code).
## Error:
If you encounter an error similar to the one below that indicates your device must be managed.
```
ERROR: AADSTS530003: Your device is required to be managed to access this resource.
Trace ID: xxxxxxxx-xxxx-xxxx-xxxxxxxx
Correlation ID: xxxxxxxx-xxxx-xxxx-xxxxxxxx
Timestamp: 2023-10-05 19:54:05Z
Interactive authentication is needed. Please run:
az login --scope https://graph.microsoft.com//.default
make: *** [Makefile:18: infrastructure] Error 1
```
### Solution
>>>>>>> 3faba43c904db668697c16df33a11c67629bd490
You will need to open your GitHub Codespaces in VSCode on your managed device. Please read more about opening your [GitHub Codespaces using VSCode](/docs/deployment/developing_in_a_codespaces.md#using-github-codespaces-in-visual-studio-code).
Binary file modified docs/process_flow.drawio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 0f8b348

Please sign in to comment.