Skip to content

Commit

Permalink
feat(discovery): Add a remote_relabel component
Browse files Browse the repository at this point in the history
This allows to receive discovery relabeling rules from a remote server
using a websocket connection. We plan this for an experimental way to
control profile collection with Grafana Cloud Profiles.
  • Loading branch information
simonswine committed Aug 21, 2024
1 parent 5f50950 commit 5e4ee53
Show file tree
Hide file tree
Showing 8 changed files with 1,366 additions and 20 deletions.
2 changes: 2 additions & 0 deletions docs/sources/reference/compatibility/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ The following components, grouped by namespace, _export_ Targets.
- [discovery.process](../components/discovery/discovery.process)
- [discovery.puppetdb](../components/discovery/discovery.puppetdb)
- [discovery.relabel](../components/discovery/discovery.relabel)
- [discovery.remote_relabel](../components/discovery/discovery.remote_relabel)
- [discovery.scaleway](../components/discovery/discovery.scaleway)
- [discovery.serverset](../components/discovery/discovery.serverset)
- [discovery.triton](../components/discovery/discovery.triton)
Expand Down Expand Up @@ -123,6 +124,7 @@ The following components, grouped by namespace, _consume_ Targets.
{{< collapse title="discovery" >}}
- [discovery.process](../components/discovery/discovery.process)
- [discovery.relabel](../components/discovery/discovery.relabel)
- [discovery.remote_relabel](../components/discovery/discovery.remote_relabel)
{{< /collapse >}}

{{< collapse title="local" >}}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
canonical: https://grafana.com/docs/alloy/latest/reference/components/discovery/discovery.remote_relabel/
description: Learn about discovery.remote_relabel
title: discovery.remote_relabel
---

<span class="badge docs-labels__stage docs-labels__item">Experimental</span>

# discovery.remote_relabel

{{< docs/shared lookup="stability/experimental.md" source="alloy" version="<ALLOY_VERSION>" >}}

`discovery.remote_relabel` accepts relabeling rules from a remote control server using Pyroscope's [settings API].
Additionally it is also possible for the control server to request the targets that are received by this component in order to show the effects of the rules at the central control server.

Multiple `discovery.remote_relabel` components can be specified by giving them different labels.

## Usage

```alloy
discovery.relabel "LABEL" {
targets = TARGET_LIST
websocket {
url = "ws://localhost:4040/settings.v1.SettingsService/GetCollectionRules?scope=alloy"
}
}
```

## Arguments

The following arguments are supported:

| Name | Type | Description | Default | Required |
| --------- | ------------------- | ------------------ | ------- | -------- |
| `targets` | `list(map(string))` | Targets to relabel | | yes |

## Blocks

The following blocks are supported inside the definition of `pyroscope.write`:

| Hierarchy | Block | Description | Required |
| ---------------------- | -------------- | --------------------------------------------------------- | -------- |
| websocket | [websocket][] | Control server settingss to. | yes |
| websocket > basic_auth | [basic_auth][] | Configure basic_auth for authenticating to the websocket. | no |

[websocket]:#websocket-block
[basic_auth]:#basic_auth-block

### websocket block

The `websocket` block describes a single command and control server. Only one `websocket` block can ben provided.

The following arguments are supported:

| Name | Type | Description | Default | Required |
| --------------------- | ---------- | ------------------------------------------------------------------ | --------- | -------- |
| `url` | `string` | Full URL to the websocket server. Needs to start with ws:// wss:// | | yes |
| `keep_alive` | `duration` | How often to sent keep alive pings. 0 to disable. | `"295s"` | no |
| `min_backoff_period` | `duration` | Initial backoff time between retries. | `"500ms"` | no |
| `max_backoff_period` | `duration` | Maximum backoff time between retries. | `"5m"` | no |
| `max_backoff_retries` | `int` | Maximum number of retries. 0 to retry infinitely. | `10` | no |

### basic_auth block

{{< docs/shared lookup="reference/components/basic-auth-block.md" source="alloy" version="<ALLOY_VERSION>" >}}

## Exported fields

The following fields are exported and can be referenced by other components:

| Name | Type | Description |
| -------- | ------------------- | --------------------------------------------- |
| `output` | `list(map(string))` | The set of targets after applying relabeling. |
| `rules` | `RelabelRules` | The currently configured relabeling rules. |

## Component health

`discovery.remote_relabel` is only reported as unhealthy when the websocket is not connected.

## Debug information

| Name | Type | Description |
| --------------------------- | ----------- | -------------------------------------------------------- |
| `websocket_status` | `string` | Status of the websocket connection. |
| `websocket_connected_since` | `time.Time` | The currently configured relabeling rules. |
| `websocket_last_error` | `string` | What was the last error when reconnecting the websocket. |

## Example

```alloy
discovery.remote_relabel "control_profiles" {
targets = [
{ "__meta_foo" = "foo", "__address__" = "localhost", "instance" = "one", "app" = "backend" },
{ "__meta_bar" = "bar", "__address__" = "localhost", "instance" = "two", "app" = "database" },
{ "__meta_baz" = "baz", "__address__" = "localhost", "instance" = "three", "app" = "frontend" },
]
websocket {
url = "wss://profiles-prod-001.grafana.net/settings.v1.SettingsService/GetCollectionRules?scope=alloy"
basic_auth {
username = env("PROFILECLI_USERNAME")
password = env("PROFILECLI_PASSWORD")
}
}
}
```

[settings API]: https://github.com/grafana/pyroscope/blob/main/api/settings/v1/setting.proto

<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components

`discovery.remote_relabel` can accept arguments from the following components:

- Components that export [Targets](../../../compatibility/#targets-exporters)

`discovery.remote_relabel` has exports that can be consumed by the following components:

- Components that consume [Targets](../../../compatibility/#targets-consumers)

{{< admonition type="note" >}}
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
Refer to the linked documentation for more details.
{{< /admonition >}}

<!-- END GENERATED COMPATIBLE COMPONENTS -->
15 changes: 10 additions & 5 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.22.5

require (
cloud.google.com/go/pubsub v1.38.0
connectrpc.com/connect v1.14.0
connectrpc.com/connect v1.16.2
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0
github.com/Azure/go-autorest/autorest v0.11.29
Expand Down Expand Up @@ -60,7 +60,7 @@ require (
github.com/grafana/cloudflare-go v0.0.0-20230110200409-c627cf6792f2
github.com/grafana/dskit v0.0.0-20240104111617-ea101a3b86eb
github.com/grafana/go-gelf/v2 v2.0.1
github.com/grafana/jfr-parser/pprof v0.0.0-20240126072739-986e71dc0361
github.com/grafana/jfr-parser/pprof v0.0.0-20240815065100-4296f63fd6d7
github.com/grafana/jsonparser v0.0.0-20240209175146-098958973a2d
github.com/grafana/kafka_exporter v0.0.0-20240409084445-5e3488ad9f9a
github.com/grafana/loki/pkg/push v0.0.0-20240514112848-a1b1eeb09583 // k201 branch
Expand Down Expand Up @@ -253,6 +253,8 @@ require (
sigs.k8s.io/yaml v1.4.0
)

require github.com/gorilla/websocket v1.5.0

require (
cloud.google.com/go v0.114.0 // indirect
cloud.google.com/go/auth v0.5.1 // indirect
Expand Down Expand Up @@ -445,11 +447,10 @@ require (
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.4 // indirect
github.com/gophercloud/gophercloud v1.12.0 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/gosnmp/gosnmp v1.37.0 // indirect
github.com/grafana/go-offsets-tracker v0.1.7 // indirect
github.com/grafana/gomemcache v0.0.0-20231204155601-7de47a8c3cb0 // indirect
github.com/grafana/jfr-parser v0.8.0 // indirect
github.com/grafana/jfr-parser v0.8.1-0.20240815065100-4296f63fd6d7 // indirect
github.com/grafana/snowflake-prometheus-exporter v0.0.0-20240813124544-9995e8354548
github.com/grobie/gomemcache v0.0.0-20230213081705-239240bbc445 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
Expand Down Expand Up @@ -601,6 +602,7 @@ require (
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/planetscale/vtprotobuf v0.6.0 // indirect
github.com/power-devops/perfstat v0.0.0-20220216144756-c35f1ee13d7c // indirect
github.com/prometheus-community/go-runit v0.1.0 // indirect
github.com/prometheus-community/prom-label-proxy v0.6.0 // indirect
Expand All @@ -615,7 +617,7 @@ require (
github.com/safchain/ethtool v0.3.0 // indirect
github.com/sagikazarmark/locafero v0.4.0 // indirect
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
github.com/samber/lo v1.38.1 // indirect
github.com/samber/lo v1.38.1
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da // indirect
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 // indirect
github.com/seccomp/libseccomp-golang v0.9.2-0.20220502022130-f33da4d89646 // indirect
Expand Down Expand Up @@ -809,3 +811,6 @@ exclude (
)

replace github.com/prometheus/procfs => github.com/prometheus/procfs v0.12.0

// TODO: Remove after https://github.com/grafana/pyroscope/pull/3416 merges and gets tagged
replace github.com/grafana/pyroscope/api => github.com/simonswine/pyroscope/api v0.0.0-20240726141528-fef95be2e3c2
20 changes: 10 additions & 10 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RX
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
code.cloudfoundry.org/clock v1.0.0/go.mod h1:QD9Lzhd/ux6eNQVUDVRJX/RKTigpewimNYBi7ivZKY8=
collectd.org v0.3.0/go.mod h1:A/8DzQBkF6abtvrT2j/AU/4tiBgJWYyh0y/oB/4MlWE=
connectrpc.com/connect v1.14.0 h1:PDS+J7uoz5Oui2VEOMcfz6Qft7opQM9hPiKvtGC01pA=
connectrpc.com/connect v1.14.0/go.mod h1:uoAq5bmhhn43TwhaKdGKN/bZcGtzPW1v+ngDTn5u+8s=
connectrpc.com/connect v1.16.2 h1:ybd6y+ls7GOlb7Bh5C8+ghA6SvCBajHwxssO2CGFjqE=
connectrpc.com/connect v1.16.2/go.mod h1:n2kgwskMHXC+lVqb18wngEpF95ldBHXjZYJussz5FRc=
dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
Expand Down Expand Up @@ -1029,10 +1029,10 @@ github.com/grafana/go-offsets-tracker v0.1.7/go.mod h1:qcQdu7zlUKIFNUdBJlLyNHuJG
github.com/grafana/gocql v0.0.0-20200605141915-ba5dc39ece85/go.mod h1:crI9WX6p0IhrqB+DqIUHulRW853PaNFf7o4UprV//3I=
github.com/grafana/gomemcache v0.0.0-20231204155601-7de47a8c3cb0 h1:aLBiDMjTtXx2800iCIp+8kdjIlvGX0MF/zICQMQO2qU=
github.com/grafana/gomemcache v0.0.0-20231204155601-7de47a8c3cb0/go.mod h1:PGk3RjYHpxMM8HFPhKKo+vve3DdlPUELZLSDEFehPuU=
github.com/grafana/jfr-parser v0.8.0 h1:/uo2wZNXrxw7tKLFwP2omJ3EQGMkD9wzhPsRogVofc0=
github.com/grafana/jfr-parser v0.8.0/go.mod h1:M5u1ux34Qo47ZBWksbMYVk40s7dvU3WMVYpxweEu4R0=
github.com/grafana/jfr-parser/pprof v0.0.0-20240126072739-986e71dc0361 h1:TtNajaiSRfM2Mz8N7ouFQDFlviXbIEk9Hts0yoZnhGM=
github.com/grafana/jfr-parser/pprof v0.0.0-20240126072739-986e71dc0361/go.mod h1:P5406BrWxjahTzVF6aCSumNI1KPlZJc0zO0v+zKZ4gc=
github.com/grafana/jfr-parser v0.8.1-0.20240815065100-4296f63fd6d7 h1:uHdCwU81kXJ7b9dMZL1ayesmT+npV0L1vriGYdbdIfs=
github.com/grafana/jfr-parser v0.8.1-0.20240815065100-4296f63fd6d7/go.mod h1:M5u1ux34Qo47ZBWksbMYVk40s7dvU3WMVYpxweEu4R0=
github.com/grafana/jfr-parser/pprof v0.0.0-20240815065100-4296f63fd6d7 h1:S+iep7nivDiFfgBD1AB8RcbW4PA7Fn1HXagHOT4MelQ=
github.com/grafana/jfr-parser/pprof v0.0.0-20240815065100-4296f63fd6d7/go.mod h1:P5406BrWxjahTzVF6aCSumNI1KPlZJc0zO0v+zKZ4gc=
github.com/grafana/jsonparser v0.0.0-20240209175146-098958973a2d h1:YwbJJ/PrVWVdnR+j/EAVuazdeP+Za5qbiH1Vlr+wFXs=
github.com/grafana/jsonparser v0.0.0-20240209175146-098958973a2d/go.mod h1:796sq+UcONnSlzA3RtlBZ+b/hrerkZXiEmO8oMjyRwY=
github.com/grafana/kafka_exporter v0.0.0-20240409084445-5e3488ad9f9a h1:jqM4NNdx8LSquKo8bPx+XWn91S2b+sgNvEcFfSJQtHY=
Expand All @@ -1059,16 +1059,12 @@ github.com/grafana/prometheus v1.8.2-0.20240514135907-13889ba362e6 h1:kih3d3M3dx
github.com/grafana/prometheus v1.8.2-0.20240514135907-13889ba362e6/go.mod h1:yv4MwOn3yHMQ6MZGHPg/U7Fcyqf+rxqiZfSur6myVtc=
github.com/grafana/pyroscope-go/godeltaprof v0.1.7 h1:C11j63y7gymiW8VugJ9ZW0pWfxTZugdSJyC48olk5KY=
github.com/grafana/pyroscope-go/godeltaprof v0.1.7/go.mod h1:Tk376Nbldo4Cha9RgiU7ik8WKFkNpfds98aUzS8omLE=
github.com/grafana/pyroscope/api v0.4.0 h1:J86DxoNeLOvtJhB1Cn65JMZkXe682D+RqeoIUiYc/eo=
github.com/grafana/pyroscope/api v0.4.0/go.mod h1:MFnZNeUM4RDsDOnbgKW3GWoLSBpLzMMT9nkvhHHo81o=
github.com/grafana/pyroscope/ebpf v0.4.7 h1:OwE7QrgEl4lH9BEzMW9NG4AySysBfCaH5fXYJiuroC8=
github.com/grafana/pyroscope/ebpf v0.4.7/go.mod h1:0iOWpGm2M6KXiP2nGa4wf02knSSjEtu11vpUOdQT5AY=
github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc h1:GN2Lv3MGO7AS6PrRoT6yV5+wkrOpcszoIsO4+4ds248=
github.com/grafana/regexp v0.0.0-20240518133315-a468a5bfb3bc/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk=
github.com/grafana/smimesign v0.2.1-0.20220408144937-2a5adf3481d3 h1:UPkAxuhlAcRmJT3/qd34OMTl+ZU7BLLfOO2+NXBlJpY=
github.com/grafana/smimesign v0.2.1-0.20220408144937-2a5adf3481d3/go.mod h1:iZiiwNT4HbtGRVqCQu7uJPEZCuEE5sfSSttcnePkDl4=
github.com/grafana/snowflake-prometheus-exporter v0.0.0-20240701202215-1d847d62ed15 h1:J4PmreN24XmbqMIKReAS/P1t7ND6NCAZApcbjBhedrY=
github.com/grafana/snowflake-prometheus-exporter v0.0.0-20240701202215-1d847d62ed15/go.mod h1:DANNLd5vSKqHloqNX4yeS+9ZRI89dj8ySZeEWlI5UU4=
github.com/grafana/snowflake-prometheus-exporter v0.0.0-20240813124544-9995e8354548 h1:XwoNrPHEl07N7EIMt/WXlzGj0Q4CDEfa+6nrdnQGOG4=
github.com/grafana/snowflake-prometheus-exporter v0.0.0-20240813124544-9995e8354548/go.mod h1:DANNLd5vSKqHloqNX4yeS+9ZRI89dj8ySZeEWlI5UU4=
github.com/grafana/tail v0.0.0-20230510142333-77b18831edf0 h1:bjh0PVYSVVFxzINqPFYJmAmJNrWPgnVjuSdYJGHmtFU=
Expand Down Expand Up @@ -1890,6 +1886,8 @@ github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/profile v1.2.1/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA=
github.com/planetscale/vtprotobuf v0.6.0 h1:nBeETjudeJ5ZgBHUz1fVHvbqUKnYOXNhsIEabROxmNA=
github.com/planetscale/vtprotobuf v0.6.0/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
Expand Down Expand Up @@ -2063,6 +2061,8 @@ github.com/sijms/go-ora/v2 v2.7.6/go.mod h1:EHxlY6x7y9HAsdfumurRfTd+v8NrEOTR3Xl4
github.com/simonpasquier/klog-gokit v0.3.0/go.mod h1:+SUlDQNrhVtGt2FieaqNftzzk8P72zpWlACateWxA9k=
github.com/simonpasquier/klog-gokit/v3 v3.4.0 h1:2eD2INbzUHuGNynPP86BCB8H6Lwfp6wlkOcuyTr3VWM=
github.com/simonpasquier/klog-gokit/v3 v3.4.0/go.mod h1:RREVB5Cc6yYHsweRfhUyM1ZP+Odb8ehxLfY8jaiqvjg=
github.com/simonswine/pyroscope/api v0.0.0-20240726141528-fef95be2e3c2 h1:7IYCsidGSiYp9mz1WIf5cavJEzZ77s8VqOOm2sEkfXw=
github.com/simonswine/pyroscope/api v0.0.0-20240726141528-fef95be2e3c2/go.mod h1:pvSSWHwrk8hPNE53ZkM6jz4k2Auq+0oQPCQ2yFb1B7A=
github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
Expand Down
1 change: 1 addition & 0 deletions internal/component/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import (
_ "github.com/grafana/alloy/internal/component/discovery/process" // Import discovery.process
_ "github.com/grafana/alloy/internal/component/discovery/puppetdb" // Import discovery.puppetdb
_ "github.com/grafana/alloy/internal/component/discovery/relabel" // Import discovery.relabel
_ "github.com/grafana/alloy/internal/component/discovery/remote_relabel" // Import discovery.remote_relabel
_ "github.com/grafana/alloy/internal/component/discovery/scaleway" // Import discovery.scaleway
_ "github.com/grafana/alloy/internal/component/discovery/serverset" // Import discovery.serverset
_ "github.com/grafana/alloy/internal/component/discovery/triton" // Import discovery.triton
Expand Down
10 changes: 5 additions & 5 deletions internal/component/common/relabel/relabel.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,13 +68,13 @@ type Regexp struct {
*regexp.Regexp
}

func newRegexp(s string) (Regexp, error) {
func NewRegexp(s string) (Regexp, error) {
re, err := regexp.Compile("^(?:" + s + ")$")
return Regexp{re}, err
}

func mustNewRegexp(s string) Regexp {
re, err := newRegexp(s)
func MustNewRegexp(s string) Regexp {
re, err := NewRegexp(s)
if err != nil {
panic(err)
}
Expand Down Expand Up @@ -122,7 +122,7 @@ type Config struct {
var DefaultRelabelConfig = Config{
Action: Replace,
Separator: ";",
Regex: mustNewRegexp("(.*)"),
Regex: MustNewRegexp("(.*)"),
Replacement: "$1",
}

Expand All @@ -131,7 +131,7 @@ func (c *Config) SetToDefault() {
*c = Config{
Action: Replace,
Separator: ";",
Regex: mustNewRegexp("(.*)"),
Regex: MustNewRegexp("(.*)"),
Replacement: "$1",
}
}
Expand Down
Loading

0 comments on commit 5e4ee53

Please sign in to comment.