Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable rsyslog duplication of systemd-journal #426

Merged
merged 4 commits into from
Jul 8, 2024

Conversation

agrare
Copy link
Member

@agrare agrare commented Nov 13, 2023

By default systemd-journal is non-persistent and rsyslog is configured to copy everything from the journal to /var/log/syslog.

If you have a /var/log/journal/ directory on persistent storage then systemd-journal will automatically persist journal files but rsyslog doesn't also automatically disable copying to syslog

TODO:

  • Live test of an rpm upgrade

Fixes #411

@Fryguy
Copy link
Member

Fryguy commented Nov 13, 2023

@bdunne Please review.

@Fryguy Fryguy added the enhancement New feature or request label Nov 13, 2023
@agrare
Copy link
Member Author

agrare commented Nov 13, 2023

Okay with the sed scriptlet here this is what imjournal module section of rsyslog.conf looks like after dnf install

#### MODULES ####

module(load="imuxsock"    # provides support for local system logging (e.g. via logger command)
       SysSock.Use="off") # Turn off message reception via local log socket; 
                          # local messages are retrieved through imjournal now.
#module(load="imjournal"            # provides access to the systemd journal
#       UsePid="system" # PID nummber is retrieved as the ID of the process the journal entry originates from
#       StateFile="imjournal.state") # File to store the position in the journal
#module(load="imklog") # reads kernel messages (the same are read from journald)
#module(load="immark") # provides --MARK-- message capability

# Provides UDP syslog reception
# for parameters see http://www.rsyslog.com/doc/imudp.html
#module(load="imudp") # needs to be done just once
#input(type="imudp" port="514")

And confirmed that after restarting rsyslog.service journal messages are no longer duplicated to /var/log/messages

@agrare
Copy link
Member Author

agrare commented Nov 13, 2023

Okay retested after adding the systemctl restart rsyslog.service line

@@ -82,6 +82,13 @@ if systemctl is-active --quiet evm-failover-monitor; then
systemctl restart evm-failover-monitor
fi

# Disable rsyslog duplicating systemd-journal output
# This will comment out the multi-line module load from /etc/rsyslog.conf
sed -i '/^module(load="imjournal"/, /StateFile="imjournal.state")/ s|^|#|' %{_sysconfdir}/rsyslog.conf
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you aren't a sed expert (which I am not), this means "starting at this regex" (/^module(load="imjournal"/) until this regex (/StateFile="imjournal.state")/), substitute the start of the line with # (s|^|#|)

@Fryguy
Copy link
Member

Fryguy commented Nov 14, 2023

And confirmed that after restarting rsyslog.service journal messages are no longer duplicated to /var/log/messages

Just to clarify. I thought this would prevent them from being written to /var/log/journal ? Only reason I'm asking is that /var/log/messages is in the settings as one of the places we collect logs from, so will this affect log collection?

@agrare
Copy link
Member Author

agrare commented Nov 14, 2023

@Fryguy we also pull logs from /var/log/journal/ and we already use journalctl on uploaded log bundles

Copy link
Member

@Fryguy Fryguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - @bdunne Please also review.

@agrare
Copy link
Member Author

agrare commented Feb 20, 2024

Hey @bdunne could you take a look at this?

@kbrock
Copy link
Member

kbrock commented Apr 5, 2024

This looks great. anything special to help get it in?

@miq-bot miq-bot added the stale label Jul 8, 2024
@miq-bot
Copy link
Member

miq-bot commented Jul 8, 2024

This pull request has been automatically marked as stale because it has not been updated for at least 3 months.

If these changes are still valid, please remove the stale label, make any changes requested by reviewers (if any), and ensure that this issue is being looked at by the assigned/reviewer(s).

agrare added 4 commits July 8, 2024 09:31
By default systemd-journal is non-persistent and rsyslog is configured
to copy everything from the journal to `/var/log/syslog`.

If you have a `/var/log/journal/` directory on persistent storage then
systemd-journal will automatically persist journal files but rsyslog
doesn't also automatically disable copying to syslog
@agrare agrare force-pushed the disable_imjournal_module_rsyslog branch from 42dda8e to f2bd3c8 Compare July 8, 2024 13:31
@agrare agrare removed the stale label Jul 8, 2024
@miq-bot
Copy link
Member

miq-bot commented Jul 8, 2024

Checked commits agrare/manageiq-rpm_build@cc27bb7~...f2bd3c8 with ruby 3.1.5, rubocop 1.56.3, haml-lint 0.51.0, and yamllint
0 files checked, 0 offenses detected
Everything looks fine. 🍰

@bdunne bdunne merged commit 7da1e9c into ManageIQ:master Jul 8, 2024
3 checks passed
@agrare agrare deleted the disable_imjournal_module_rsyslog branch July 8, 2024 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Disable rsyslog/journald log copies on appliances
5 participants