Skip to content

Commit

Permalink
Minor format
Browse files Browse the repository at this point in the history
  • Loading branch information
zhiheng-huang committed Jan 18, 2025
1 parent 9f48173 commit db13ff9
Showing 1 changed file with 11 additions and 7 deletions.
18 changes: 11 additions & 7 deletions denser_retriever/experiments/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,24 @@ We note that the ingestion stops when it exceeds the following quotas specified
in `denser_retriever/configs/cost_config.json`.

```json
"es_storage_quota_gb": 2.0,
"vector_storage_quota_gb": 2.0,
"vector_token_quota_million": 2.0
{
"es_storage_quota_gb": 2.0,
"vector_storage_quota_gb": 2.0,
"vector_token_quota_million": 2.0
}
```

The ingestion storage and vector tokens and their costs are reported after the ingestion. If you run a large experiment,
make sure to set the quotas to a higher value. As a reference, the ingestion of scifact dataset (5183 documents) has the
following ingestion stats:

```json
'num_docs': 5183,
'es_storage_gb': 0.03670822083950042,
'vector_storage_gb': 0.014828681945800781,
'vector_tokens': 1635249
{
"num_docs": 5183,
"es_storage_gb": 0.03670822083950042,
"vector_storage_gb": 0.014828681945800781,
"vector_tokens": 1635249
}
```

## Retrieving
Expand Down

0 comments on commit db13ff9

Please sign in to comment.