Skip to content

Commit

Permalink
Include documentation for Collector API datastore
Browse files Browse the repository at this point in the history
  • Loading branch information
zakkak committed Oct 1, 2024
1 parent ee08896 commit 910a861
Show file tree
Hide file tree
Showing 7 changed files with 108 additions and 2 deletions.
2 changes: 1 addition & 1 deletion docs/site/content/en/docs/Concepts/core-concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ A `Function` is a JavaScript function that is executed on the server side. Funct

## Datasource

A `Datasource` is a **required** top-level organizational construct that defines the source of the data to be stored or retrieved by Horreum. Currently, Horreum supports 2 types of `Datasource`: Postgres and Elasticsearch
A `Datasource` is a **required** top-level organizational construct that defines the source of the data to be stored or retrieved by Horreum. Currently, Horreum supports 3 types of `Datasource`: Postgres, Elasticsearch, and [Collector](https://github.com/Karm/collector)

## Baseline

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
43 changes: 43 additions & 0 deletions docs/site/content/en/docs/Integrations/collector/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
title: Collector API
date: 2024-10-01
description: Use Collector API to query JSON for analysis
categories: [Integration, Datasource]
weight: 1
---

If you have a lot of data already stored in a [Collector](https://github.com/Karm/collector) instance, you can query it and analyze the data for regressions in Horreum.

## Configuration

To configure a test to use the `Collector API` backend, you need to be a team administrator. With the correct permissions, you can:

1. Generate a new API key for the `Collector API` backend: Please see the collector docs on how to [Create a new API token](https://github.com/Karm/collector?tab=readme-ov-file#create-a-new-api-token)
2. Navigate to `Administration` -> `Datastores` configuration page, e.g. `http://localhost:8080/admin#datastores`
2. Select the `Team` from the `Team` dropdown that you wish to configure
2. Click `New Datastore`
{{% imgproc new-datastore Fit "1115x469" %}}
New Datastore
{{% /imgproc %}}
3. Configure the `Collector API` Datastore:
{{% imgproc modal Fit "1115x469" %}}
New Collector API Datastore
{{% /imgproc %}}
1. Select `Collector API` from the `Datastore Type` dropdown
2. Provide a `Name` for the Datastore
3. Enter the `URL` for the Collector instance
4. Enter the `API Key` for the Collector instance, generated in step 1
5. Click `Save`

## Test Configuration

To configure a test to use the `Collector API` backend, you can:

1. Navigate to a test configuration page, e.g. `http://localhost:8080/test/10`
2. Select the `Collector API` backend defined in the `Datastores` configuration from the `Datastore` dropdown

{{% imgproc configure-test Fit "1115x469" %}}
Configure Test
{{% /imgproc %}}

3. Click `Save`
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
63 changes: 63 additions & 0 deletions docs/site/content/en/docs/Tutorials/query-collector/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
title: Query Collector API
date: 2024-10-01
description: Query JSON data from Collector and analyze it with Horreum
categories: [Tutorial]
weight: 3
---

> **Prerequisites**:
> 1. Horreum is running, and you are logged in
> 2. You have access to a running [Collector](https://github.com/Karm/collector) instance that already contains JSON data
> 3. You have previously defined a `Schema` for the JSON data you wish to analyze, please see [Define a Schema](/docs/tasks/define-schema-and-views/)
## Create a Test and query data from a Collector instance

This tutorial will guide you through how to connect to a remote [Collector](https://github.com/Karm/collector) instance, and perform change detection on existing data in an index.

## Configure Collector Datastore

Please follow the [Collector Integration](/docs/integrations/collector/) guide to configure a new Collector Datastore.

## Query Data from Collector

The procedure is the same as described in the [Upload your first Run](/docs/tutorials/create-test-run/) tutorial

To query data from a Collector instance, you need to know the `tag` and `imgName` of the data you wish to analyze.
You will also need to determine the date range of the data you wish to analyze using `newerThan` and `olderThan`.

```json
{
"tag": "quarkus-main-ci",
"imgName": "quarkus-integration-test-main-999-SNAPSHOT-runner",
"newerThan": "2024-09-20 00:00:00.000",
"olderThan": "2024-09-25 00:00:00.000"
}
```

where;

- **tag**: the tag of the data you wish to analyze
- **imgName**: the image name (aka test) of the data you wish to analyze
- **newerThan**: the start date of the data you wish to analyze
- **olderThan**: the end date of the data you wish to analyze

The query can be executed by making a call to the Horreum API;

```bash
$ curl 'http://localhost:8080/api/run/data?test='$TEST'&start='$START'&stop='$STOP'&owner='$OWNER'&access='$ACCESS \
-s -H 'content-type: application/json' -H 'Authorization: Bearer '$TOKEN \
-d @/tmp/collector_query.json
```

The query will return a list of `RunID`'s for each json object retrieved and analyzed from Collector.

## What Next?

After successfully querying data from Collector, you can now:
- optionally [Transform Runs to Datasets](/docs/tasks/trasnform-runs-to-datasets/) to transform the data into datasets
- [Configure Change Detection](/docs/tasks/configure-change-detection/) to detect regressions in the data.
- [Configure Actions](/docs/tasks/configure-actions/) to trigger events when regressions are detected.
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ weight: 3
## Create a Test and query data from an Elasticsearch instance

This tutorial will present guide you through how to connect to a remote Elasticsearch instance, and perform change detection on existing data in an index.
This tutorial will guide you through how to connect to a remote Elasticsearch instance, and perform change detection on existing data in an index.

## Configure Elasticsearch Datastore

Expand Down

0 comments on commit 910a861

Please sign in to comment.