Skip to content
192 changes: 192 additions & 0 deletions content/nginx-one/agent/metrics/configure-otel-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,195 @@ You can validate that metrics are successfully exported by using the methods bel
```text
Everything is ready. Begin running and processing data.
```

### Custom OTel Configuration

NGINX Agent generates a default OpenTelemetry config (located at `/etc/nginx-agent/opentelemetry-collector-agent.yaml`) to send metrics to your management plane. An option is provided to
bring your own [OpenTelemetry configs](https://opentelemetry.io/docs/collector/configuration/) which will be merged with the NGINX Agent default config.

The **order of the OpenTelemetry config files matters**: the last config in the list will take priority over others listed, if they have the same value configured.

To have NGINX Agent use your own OpenTelemetry config:

1. Edit the configuration file as root `vim /etc/nginx-agent/nginx-agent.conf`
2. Add the collector property
3. Save and restart the NGINX agent service `sudo systemctl restart nginx-agent`
Comment on lines +52 to +54
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Edit the configuration file as root `vim /etc/nginx-agent/nginx-agent.conf`
2. Add the collector property
3. Save and restart the NGINX agent service `sudo systemctl restart nginx-agent`
1. Edit the configuration file as root `vim /etc/nginx-agent/nginx-agent.conf`
1. Add the collector property
1. Save and restart the NGINX agent service `sudo systemctl restart nginx-agent`

Automatic Hugo list enumeration.


```yaml
collector:
additional_config_paths:
- "/etc/nginx-agent/my_config.yaml"
```

#### Example usage

{{< call-out "important" >}} NGINX Agent uses `/default` for naming its default processors, exporters and pipelines using the same naming in your own config might cause issues with sending metrics to your management plane {{< /call-out >}}

- **Add Prometheus Exporter**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Add Prometheus Exporter**
To add a Prometheus exporter:

Needless list indentation, and bold pseudo-heading.

The latter would be picked up by Markdown linting.


{{< tabs name="prometheus-exporter" >}}
{{% tab name="NGINX Plus" %}}

```yaml
exporters:
prometheus:
endpoint: "127.0.0.1:5643"
resource_to_telemetry_conversion:
enabled: true
namespace: test-space

service:
pipelines:
metrics/prometheus-example-pipeline:
receivers:
- nginxplus
processors:
- resource/default
exporters:
- prometheus
```

{{% /tab %}}
{{% tab name="NGINX OSS" %}}

```yaml
exporters:
prometheus:
endpoint: "127.0.0.1:5643"
resource_to_telemetry_conversion:
enabled: true
namespace: test-space

service:
pipelines:
metrics/prometheus-example-pipeline:
receivers:
- nginx
processors:
- resource/default
exporters:
- prometheus
```

{{% /tab %}}
{{< /tabs >}}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

Excess empty line.


- **Third-party OTel Collector**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Third-party OTel Collector**
To add a third-party OpenTelemetry Collector:

Needless list indentation, and bold pseudo-heading.

The latter would be picked up by Markdown linting.


{{< tabs name="third-party-collector" >}}
{{% tab name="NGINX Plus" %}}

```yaml
exporters:
otlp/local-collector:
endpoint: "my-local-collector.com:443"
timeout: 10s
retry_on_failure:
enabled: true
initial_interval: 10s
max_interval: 60s
max_elapsed_time: 10m
tls:
insecure: true
Comment on lines +125 to +132
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
timeout: 10s
retry_on_failure:
enabled: true
initial_interval: 10s
max_interval: 60s
max_elapsed_time: 10m
tls:
insecure: true


service:
pipelines:
metrics/otlp-example-pipeline:
receivers:
- nginxplus
processors:
- resource/default
exporters:
- otpl/local-collector
```

{{% /tab %}}
{{% tab name="NGINX OSS" %}}

```yaml
exporters:
otlp/local-collector:
endpoint: "my-local-collector.com:443"
timeout: 10s
retry_on_failure:
enabled: true
initial_interval: 10s
max_interval: 60s
max_elapsed_time: 10m
tls:
insecure: true
Comment on lines +152 to +159
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
timeout: 10s
retry_on_failure:
enabled: true
initial_interval: 10s
max_interval: 60s
max_elapsed_time: 10m
tls:
insecure: true


service:
pipelines:
metrics/otlp-example-pipeline:
receivers:
- nginx
processors:
- resource/default
exporters:
- otpl/local-collector
```

{{% /tab %}}
{{< /tabs >}}


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

Excess empty line

- **Add Debug Exporter**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Add Debug Exporter**
To add a debug exporter:

Needless list indentation, and bold pseudo-heading.

The latter would be picked up by Markdown linting.


{{< tabs name="debug-exporter" >}}
{{% tab name="NGINX Plus" %}}

```yaml
exporters:
debug:
verbosity: detailed
sampling_initial: 5
sampling_thereafter: 200

service:
pipelines:
metrics/debug-example-pipeline:
receivers:
- nginxplus
processors:
- resource/default
exporters:
- debug
```

{{% /tab %}}
{{% tab name="NGINX OSS" %}}

```yaml
exporters:
debug:
verbosity: detailed
sampling_initial: 5
sampling_thereafter: 200

service:
pipelines:
metrics/debug-example-pipeline:
receivers:
- nginx
processors:
- resource/default
exporters:
- debug
```

{{% /tab %}}
{{< /tabs >}}


Comment on lines +222 to +223
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

Excess empty lines


### Troubleshooting

To view the merged OpenTelemetry configuration set the NGINX Agent log level to "debug" in `/etc/nginx-agent/nginx-agent.conf`, and restart NGINX Agent:

1. Edit the configuration file `sudo vim /etc/nginx-agent/nginx-agent.conf`
1. Change the log property
```yaml
log:
level: debug
Comment on lines +227 to +233
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To view the merged OpenTelemetry configuration set the NGINX Agent log level to "debug" in `/etc/nginx-agent/nginx-agent.conf`, and restart NGINX Agent:
1. Edit the configuration file `sudo vim /etc/nginx-agent/nginx-agent.conf`
1. Change the log property
```yaml
log:
level: debug
To view the merged OpenTelemetry configuration, change the NGINX Agent log level to "debug" in `/etc/nginx-agent/nginx-agent.conf`:
```yaml
log:
level: debug
```
Then restart NGINX Agent.

Not being prescriptive about editor, removing the list since it's just 2 items.