Graphs intermittently fail to load

[Steering7253]

Support guidelines

• I've read the support guidelines

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Same as #124 .

Graphs intermittently fail to load when using rrdcached on the latest docker image ( 24.9.0 - Tue Oct 15 2024 13:15:45 GMT-0600).

Disabling rrdcached resolves it (but of course, adds more load to an already heavily loaded drive).

This is not a poller issue or anything else. LibreNMS is attempting for unknown reasons to display a non-existent graph for a port that is and always has been down.

Expected behaviour

Graphs load

Actual behaviour

Graphs don't load and show an error about a non-existent file not existing, on about half of all graph loads.

Clicking the graph takes you to another page with another ~50% chance of graphs loading (most of them don't, most of the time).

Steps to reproduce

1. Install docker image
2. Set up rrdcached
3. Refresh the Devices page, mouse over a device, and look at the popup

Docker info

Client: Docker Engine - Community
 Version:    27.3.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.17.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.7
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 21
  Running: 19
  Paused: 0
  Stopped: 2
 Images: 119
 Server Version: 27.3.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc version: v1.1.14-0-g2c9f560
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-124-generic
 Operating System: Ubuntu 22.04.5 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.624GiB
 Name: cluster5
 ID: ZNIZ:SZNF:4QEL:JG2B:EEJM:Y2UE:356G:ZEEN:RDKK:AVLD:T2EE:TB3T
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Docker Compose config

No.

  rrdcached:
    image: crazymax/rrdcached
    container_name: librenms_rrdcached
    volumes:
      - "./librenms/rrd:/data/db"
      - "./librenms/journal:/data/journal"
    environment:
      - "TZ=${TZ}"
      - "PUID=${PUID}"
      - "PGID=${PGID}"
      - "LOG_LEVEL=LOG_INFO"
      - "WRITE_TIMEOUT=1800"
      - "WRITE_JITTER=1800"
      - "WRITE_THREADS=4"
      - "FLUSH_DEAD_DATA_INTERVAL=3600"
    restart: always

  librenms:
    entrypoint: /data/entrypoint.sh
    image: librenms/librenms:latest
    container_name: librenms
    hostname: librenms
    cap_add:
      - NET_ADMIN
      - NET_RAW
    ports:
      - target: 8000
        published: 8000
        protocol: tcp
    depends_on:
      - db
      - redis
      - msmtpd
      - rrdcached
    volumes:
      - "./librenms:/data"
    env_file:
      - "./librenms.env"
    environment:
      - "TZ=${TZ}"
      - "PUID=${PUID}"
      - "PGID=${PGID}"
      - "DB_HOST=db"
      - "DB_NAME=${MYSQL_DATABASE}"
      - "DB_USER=${MYSQL_USER}"
      - "DB_PASSWORD=${MYSQL_PASSWORD}"
      - "DB_TIMEOUT=60"
    restart: always

Logs

Nothing relevant.

Additional info

No response

[dot-mike]

What did you set the value of rrdcached in the librenms config to? Navigate to your compose directory and execute the following: docker compose exec -T librenms lnms config:get rrdcached and paste the output here.

I suspect that you didn't configure this value properly to point to the container that runs rrdcached. If the output of the previous command has unix:/run/... something then it's wrong. Correct value for you would be: librenms_rrdcached:42217 this can be fixed by executing the command docker compose exec -T librenms lnms config:set rrdcached "librenms_rrdcached:42217".