How to parse Netgear Switch Syslog messages?

[AidanNelson]

Hi There,

I'm having some issues parsing syslog messages from Netgear Network switches and would appreciate any guidance on the correct parser setup.

The syslog messages are formatted as such:

<14> Apr 25 16:43:29 PAA-SW1-1 General[procLOG]: main.c(257) 272264 %% Stopping System API  application

So far I've tried using the Syslog parser and the Regex parser to no avail. Here is what I've tried and the error messages I'm seeing:

Syslog Parser

# /etc/td-agent/td-agent.conf
<source>
  @type syslog
  port 5140
  bind 0.0.0.0
  tag system
  <parse>
    @type syslog
    time_format "%b %d %H:%M:%S"
  </parse>
</source>

This resulted in the following error:

invalid input data="<13> Apr 25 16:20:08 PAA-SW2-1 TRAPMGR[ipMapForwarding]: traputil.c(795) 267495 %% Address conflict detected for IP: 10.101.0.1. Conflicting Host MAC: 68:3a:1e:42:18:ed\x00" error_class=Fluent::TimeParser::TimeParseError error="invalid time format: value =  Apr 25 16:20:08, error_class = ArgumentError, error = string doesn't match"

I also tried adding a space at the start of the time format because the source syslog message seems to deviate from RFC-3164 Log
specification in that respect. This looked like time_format " %b %d %H:%M:%S"and resulted in my messages being totally ignored.

Regex Parser

# /etc/td-agent/td-agent.conf
<source>
  @type syslog
  port 5140
  bind 0.0.0.0
  tag system
  <parse>
    @type regexr
    expression /^\<(?<pri>[0-9]+)\> (?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[^ :\[]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
    time_format "%b %d %H:%M:%S"
  </parse>
</source>

Note the added space between the match group and the match group (added because Netgear includes those spaces.

This resulted in the following error message:

failed to parse message data="<14> Apr 25 16:47:26 PAA-SW1-1 General[procLOG]: common.c(324) 272321 %% Starting System API example application\x00"

Any thoughts on how I could parse this message?

Thanks!

[kenhys]

@AidanNelson

If you use with @regexp parser, need to consider that pri information is consumed by syslog input plugin.
so, maybe to parse incoming message, need to fix expression to handle that.

  <parse>
     @type regexp
     expression /^ (?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[^ :\[]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
      time_format "%b %d %H:%M:%S"
  </parse>

drawback is that pri is lost in this case.

The following configuration is better to keep pri.

severity_key pri
<parse>
  @type regexp
  expression /^ (?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[^ :\[]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
  time_format "%b %d %H:%M:%S"
</parse>

[daipom]

pri consists of Facility and Severity. (pri = 8 * Facility + Severity)

• ref: https://www.rfc-editor.org/rfc/rfc3164#section-4.1.1

For example, <13> means Facility: 1 (user-level messages) and Severity: 5 (Notice).

So in order to fully keep pri, we should set facility_key as well.

severity_key severity
facility_key facility
<parse>
  @type regexp
  expression ...
  time_format ...
</parse>

[daipom]

I wondered if there is documentation on the specification that we don't have to include PRI in the regex.

There is actually the documentation!

• https://docs.fluentd.org/input/syslog#less-than-parse-greater-than-directive

This says the parser receives the text without PRI.

In addition, it does not matter to the parser whether the PRI information is included in the record or not.
By default, PRI is used only for tag.
If we need to include it in the records, we need set severity_key or facility_key regardless of the parser setting.

[kenhys]

@AidanNelson

If you want to extract pri, it may better to use @type tcp input plugin with @type regexp parser plugin. (or @type udp and @type regexp parser plugin)``

Another solution like this:

<source>
  @type tcp
  ...
  <parse>
    @type regexp
    expression /^\<(?<pri>[0-9]+)\> (?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[^ :\[]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$/
    time_format "%b %d %H:%M:%S"
  </parse>
</source>

[daipom]

Hmm, while there are workarounds, this raises the question of whether in_syslog's specifications are correct.

The rfc3164 syslog format consists of PRI, HEADER, and MSG. (https://www.rfc-editor.org/rfc/rfc3164#section-4.1)

The focus is on whether we should allow space between PRI and HEADER.

• Recommended example: <14>timestamp hostname tag msg
• Gray area: <14> timestamp hostname tag msg (with space between PRI and HEADER)

The current in_syslog considers the latter invalid, but we might as well consider this valid.

https://www.rfc-editor.org/rfc/rfc3164#section-4.1.2 says:

The TIMESTAMP will immediately follow the trailing ">" from the PRI part

"will" is used, not "must". (If there is no other possibility, "must" should be used.)
It doesn't seem to me that rfc3164 is ruling out the possibility of space between PRI and HEADER.

[daipom]

Now I created the issue

• in_syslog: consider space between PRI and HEADER valid for RFC3164 #4158

[daipom]

@AidanNelson

I think the best solution for this issue is using time_format " %b %d %H:%M:%S", as follows.
(Solve the space between PRI and HEADER as time_format)

<source>
  @type syslog
  tag system
  <parse>
    @type syslog
    time_format " %b %d %H:%M:%S"
  </parse>
</source>