Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kinda big delay #1

Open
KillzTheSpy2 opened this issue Jun 1, 2024 · 1 comment
Open

kinda big delay #1

KillzTheSpy2 opened this issue Jun 1, 2024 · 1 comment
Labels
help wanted Extra attention is needed

Comments

@KillzTheSpy2
Copy link

i thought it would be in realtime.
It refreshes about every second and i've tried everything.
I've changed the grafana config to refresh every 100ms, even the influxdb and telegraph configs. Nothing changes the refresh rate.
Any help?

@fabiomix
Copy link
Owner

fabiomix commented Jun 4, 2024

Hi KillzTheSpy2.

This project has been developed and tested on a low-end notebook (dual-core 2.0GHz, 4GB RAM, non-SSD disk) and it can collect one metric every second just fine. The primary scope of the project is to collect and STORE telemetry data, and I thought that the current settings for Telegraf can achieve this purpose. Maybe with different hardware it's possible to push these limits and have better results.

To increase data frequency I can suggest (if you haven't already tried these) to:

  • decrease Telegraf interval in telegraf.conf
  • decrease Telegraf precision in telegraf.conf
  • decrease Telegraf flush_interval (or metric_batch_size) in telegraf.conf
  • decrease Grafana min_refresh_interval in docker-compose
  • add shorter refresh_intervals in the Grafana dashboard

Be aware of the "[outputs.influxdb] did not complete within its flush interval" error if Telegraf output is faster than InfluxDB input.

I'm still trying in my spare time to optimize this setup for better performance and even if I would like - one day - to replace the Python script with GoLang, I think that the bottleneck here is InfluxDB (UNCONFIRMED). I didn't see big improvements with InfluxDB v2.

If you don't need to store data, you can switch to WebSocket. Grafana can receive WebSocket and Telegraf has an output plugin for it. See Stream metrics from Telegraf to Grafana.

This bypasses InfluxDB completely. Even on a low-end PC, you should receive 60 packets/sec in Grafana, which is almost real-time I would say, but you need to rewrite the dashboard queries.

This solution has no storage, and I can't/won't (for now) remove InfluxDB from the stack since this project started as an excuse to play with InfluxDB.

@fabiomix fabiomix added the help wanted Extra attention is needed label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants