Suggested log collection?

[danthegoodman1]

Wondering what is used in practice, and/or what would be suggested for collecting system logs.

Initially I am thinking having a container of https://vector.dev/docs/reference/configuration/sources/docker_logs/ running on all hosts would be a great start, but not sure how mrsk might be manipulating logs and/or permitting access, etc.

[calmyournerves]

Are you referring to the host logs (syslog, journald etc.) or the logs of the Docker containers?

#131 added support for configuring the logging driver used by Docker. In our case, we'll use the awslogs driver which sends all container logs to CloudWatch.

[danthegoodman1]

logs of Docker containers, seems I am blind! Does the default configuration handle rotating logs and such?

[danthegoodman1]

Nevermind I see json is used by default not local. Alright got a path from here, thanks! I do see a max size though!

[calmyournerves]

For the json-file driver, you can specify max-size and/or max-file to enable automatic rotation. If nothing is configured for MRSK, it will pass max-size=10m to all containers (which enables log rotation as well, as the Docker default for max-file is 1).

Please also note that the PR linked above has been merged but not released yet.

[danthegoodman1]

Thanks for the heads up. One thing I am thinking is that if I use json-logs and run something like vector.dev inside a container it won't be able to see the logs of other containers. I'm thinking I either need to do some docker socket exposure or need to install vector on the hosts directly instead.

Following https://vector.dev/docs/setup/installation/platforms/docker/ I am curious if there are any recommended methods for placing configuration files (and updating them) on the hosts on every deploy. Didn't seem to find any in the README.

[danthegoodman1]

In theory I could wrap that container and just write the file to the container filesystem with a tiny Dockerfile.

[christianbpedersen]

Should be fairly simple to include a Mezmo container on each host, if that's your thing.