Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Influxdb metrics pushing error #271

Open
Hacker-web-Vi opened this issue Nov 14, 2024 · 1 comment
Open

Influxdb metrics pushing error #271

Hacker-web-Vi opened this issue Nov 14, 2024 · 1 comment
Assignees

Comments

@Hacker-web-Vi
Copy link

Hacker-web-Vi commented Nov 14, 2024

Hello!
I'm trying to push zgs_node metrics to the influxdb bucket, but it doesn't work.
Influxdb is up and running. I tried to upload some metrics manually using the same auth method as provided in the config below and it worked. There is no related error in the log.

zgs_node v0.7.4

#######################################################################
###                      Metrics Options                            ###
#######################################################################

[metrics]

# Whether to enable metrics.
enabled = true

# Interval to output metrics periodically, e.g. "10s", "30s" or "60s".
report_interval = "10s"

# File name to output metrics periodically.
file_report_output = "metrics.csv"

# Influxdb configurations to output metrics periodically.
influxdb_report_host = "http://168.119.10.134:8416"
influxdb_report_db = "0g-test"
influxdb_report_username = "0g-test"
influxdb_report_password = "test-0-0-0"

# Storage node name as a tag.
influxdb_report_node = "0g-test"

In trace logs I was able to find our influxdb host

0g-testnet-validator@trusted-point-1:~/.storage-node$ tail -f /home/0g-testnet-validator/.storage-node/log/zgs.log.$(TZ=UTC date +%Y-%m-%d) | grep 168.119.10.134
2024-11-14T19:47:17.073208Z TRACE hyper::client::pool: take? ("http", 168.119.10.134:8416): expiration = Some(90s)
2024-11-14T19:47:17.073216Z DEBUG hyper::client::pool: reuse idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:17.227662Z TRACE hyper::client::pool: put; add idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:17.227665Z DEBUG hyper::client::pool: pooling idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:31.735756Z TRACE hyper::client::pool: take? ("http", 168.119.10.134:8416): expiration = Some(90s)
2024-11-14T19:47:31.735774Z DEBUG hyper::client::pool: reuse idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:31.898813Z TRACE hyper::client::pool: put; add idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:31.898818Z DEBUG hyper::client::pool: pooling idle connection for ("http", 168.119.10.134:8416)
2024-11-14T19:47:39.209178Z TRACE hyper::client::pool: take? ("http", 168.119.10.134:8416): expiration = Some(90s)
@Hacker-web-Vi Hacker-web-Vi changed the title influxdb metricspushing error Influxdb metrics pushing error Nov 14, 2024
@boqiu
Copy link
Contributor

boqiu commented Nov 15, 2024

Basically, file_report_output is used for dev purpose, and need not to configure it.

As for reporting to influxdb, is there any stderr or logs? E.g.

  • stderr: Exit metrics reporting due to error
  • debug log: failed to write points to influxdb

Moreover, you could check the metrics via RPC:

curl -X POST --data '{"jsonrpc":"2.0","method":"admin_getMetrics","params":["metrics"],"id":1}' -H "Content-Type: application/json" http://127.0.0.1:5679 | jq -C ".result"

There is a metric metrics_report_failures records the number of failures to report metrics to influcdb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants