After going through multiple sources and reading multiple issues, I somewhat managed to run cAdvisor in Rootless Podman.
But I'm facing the same issues as mentioned in this comment in issue #2424 where cAdvisor is working but the name of the containers are not present.

Running cAdvisor

I followed the docs and also went through Cub0n's guide, this github gist and many issues reported in this repo for guidance on how to make cAdvisor work for rootless podman.

I'm sharing all the knowledge I gained from my attempt to make it work so that others can be benefited. Command/s used to run cAdvisor with rootless podman

podman run \
 --privileged \
 --restart unless-stopped \
 -p 8080:8080 \
 --device /dev/kmsg \
 --volume /:/rootfs:ro \
 --volume /dev/disk/:/dev/disk:ro \
 --volume /etc/machine-id:/etc/machine-id:ro \
 --volume /sys:/sys:ro \
 --volume /sys/fs/cgroup:/sys/fs/cgroup:ro \
 --volume /var/lib/dbus/machine-id:/var/lib/dbus/machine-id:ro \
 --volume /var/lib/containers:/var/lib/containers:ro `# Root containers` \
 --volume /run/user/$(id -u)/podman:/var/run/podman:ro \
 --name "cadvisor" \
 ghcr.io/cadvisor/cadvisor:latest \
--podman="unix:///var/run/podman/podman.sock"\
 --docker="unix://" \
--housekeeping_interval=10s \
--docker_only=true

Command Explanation

• Ignore the housekeeping_interval and docker_only flag those are just for optimization. If you want to read more, here is the source.
• If you have installed podman-docker on your system then --docker flag is to be set with no socket path or empty path as suggested in this comment .
• The --podman flag expects the path to the podman socket. If you observe carefully I have attached /run/user/$(id -u)/podman this path to /var/run/podman on the container so the socket path on the container is /var/run/podman/podman.sock.
• Other volume mounts and device mounts are as stated in docs but with some modifications for rootless containers.

Output

On running the above command I got this error. Below is just 2 lines of endless error.

E1125 20:28:20.065450       1 manager.go:1116] Failed to create existing container: /user.slice/user-1001.slice/[email protected]/app.slice/dumb.service/libpod-payload-7cae8bcd92c4a270d9e3f4ec2c1ee80c6a1cc42bc32ded28ed2de5162337cafd: unable to determine 7cae8bcd92c4a270d9e3f4ec2c1ee80c6a1cc42bc32ded28ed2de5162337cafd rw layer id
W1125 20:28:24.317558       1 manager.go:1169] Failed to process watch event {EventType:0 Name:/user.slice/user-1001.slice/[email protected]/app.slice/6f5b2c27534db8f2dc0350b46a5faabd00961fa9f5746b9e82bf1e05bc5e3a71.service WatchSource:0}: unable to determine 6f5b2c27534db8f2dc0350b46a5faabd00961fa9f5746b9e82bf1e05bc5e3a71 rw layer id

Solution

I found out that attaching /sys/fs/cgroup to the container was giving me this error. So, I ran the above command without attaching this path to container and it was working good.

Final Command

podman run \
 --privileged \
 --restart unless-stopped \
 -p 8080:8080 \
 --device /dev/kmsg \
 --volume /:/rootfs:ro \
 --volume /dev/disk/:/dev/disk:ro \
 --volume /etc/machine-id:/etc/machine-id:ro \
 --volume /sys:/sys:ro \
 --volume /var/lib/dbus/machine-id:/var/lib/dbus/machine-id:ro \
 --volume /var/lib/containers:/var/lib/containers:ro `# Root containers` \
 --volume /run/user/$(id -u)/podman:/var/run/podman:ro \
 --name "cadvisor" \
 gcr.io/cadvisor/cadvisor:latest \
--podman="unix:///var/run/podman/podman.sock"\
 --docker="unix://" \
--housekeeping_interval=10s \
--docker_only=true

Results

As mentioned earlier, I'm unable to get the name of the containers in metrics. All is see in metrics is (Below is just 4 lines of bigger sample)

container_fs_inodes_free{device="/rootfs/home/legion/.local/share/containers/storage/overlay-containers/104af02e10687d3945e305d5c951eb38aa3bfcec1d353fa8e2820bd88adec349/userdata/shm",id="/"} 2.043481e+06 1732567259966
container_fs_inodes_free{device="/rootfs/home/legion/.local/share/containers/storage/overlay-containers/14e900ba2610e21eb96a6afce6f23f8d22896cf3313eb437904205d18b1e3857/userdata/shm",id="/"} 2.043481e+06 1732567259966
container_fs_inodes_free{device="/rootfs/home/legion/.local/share/containers/storage/overlay-containers/15c9c84d9e55a8ad05a336194b5818e105600ffd08854691a6ad65cdb5e4f992/userdata/shm",id="/"} 2.043481e+06 1732567259966
container_fs_inodes_free{device="/rootfs/home/legion/.local/share/containers/storage/overlay-containers/1b9b81dbb2e801d6d2177ba33ce20b3fb07e5fcce4d6188a7de5657624319e35/userdata/shm",id="/"} 2.043481e+06 1732567259966

Root vs Rootless File Locations

/var/lib/containers

Attach the following path to the container:

• Rootless Containers = $HOME/.local/share/containers. Example:

--volume=$HOME/.local/share/containers:/var/lib/containers:ro `# Rootless Containers` \

• Root Containers = /var/lib/containers. Example:

--volume /var/lib/containers:/var/lib/containers:ro `# Root containers`  \

Socket

Socket has been mentioned above, for root no need to attach it using --volume flag. In rootless operation we are overriding the root socket with user socket.

Please help me out with this. Thanks