Add ability to send syslog RFC5424 structured_data

[mrdavidlaing]

Currently the syslog output always sets the syslog RFC5424 structured data field to -.

The PR adds the ability to send customised syslog RFC5424 structured data for the syslog output.

This is really helpful if you want to pass additional "meta data" along with your syslog message.

ie.

syslog {
    host => "10.244.10.6"
    port => 514
    rfc => "rfc5424"
    facility => "user-level"
    severity => "informational"
    structured_data => 'LOGSTASH@42 extrafield="%{[@extrafield]}"'
}

will result in syslog messages that look this this:

<14>1 2015-01-16T11:49:18.670+0000 logsearch-workspace LOGSTASH - - [LOGSTASH@42 extrafield="extrafield_value"] 75.71.143.76, 10.10.2.122 - - - [13/Jan/2015:15:19:05 +0000] "GET /favicon.ico HTTP/1.1" 404 199

Omitting the structured_data config

syslog {
    host => "10.244.10.6"
    port => 514
    rfc => "rfc5424"
    facility => "user-level"
    severity => "informational"
}

Keeps the existing behaviour of sending -

<14>1 2015-01-16T11:49:18.670+0000 logsearch-workspace LOGSTASH - - - 75.71.143.76, 10.10.2.122 - - - [13/Jan/2015:15:19:05 +0000] "GET /favicon.ico HTTP/1.1" 404 199

[mrdavidlaing]

@jordansissel - could you give me some pointers on what I need to do to get this PR ready for merging?

Thanks!

[RanjodhKaur]

Hi

as you mentioned we can get some value in structure data by using:

syslog {
host => "10.244.10.6"
port => 514
rfc => "rfc5424"
facility => "user-level"
severity => "informational"
structured_data => 'LOGSTASH@42 extrafield="%{[@extrafield]}"'
}

In which file we need to add this?

[mrdavidlaing]

@RanjodhKaur,

Just add the extra data you want to the structured_data field value.

ie, if you wanted to add some extra data about the environment you are sending data from:

syslog {
    host => "10.244.10.6"
    port => 514
    rfc => "rfc5424"
    facility => "user-level"
    severity => "informational"
    structured_data => 'LOGSTASH@42 environment="PRODUCTION-EU-WEST"'
}

[purbon]

Thanks a lot for your contribution @mrdavidlaing, in order to move forward with your PR is going to be necessary to sign the CLA agreement, you can find more information at https://www.elastic.co/contributor-agreement.

On the other side, it would be super nice if you can add test for this change. We try to enforce that all PR introduce some kind of testing, so we're able to increase the overall quality. Don't hesitate to ask any question regarding you might have, more than looking forward to help.

Looking forward to get this in,

/cheers

[elasticsearch-release]

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.

[ejholmes]

👍 Would love to have this!

[breml]

Hi @mrdavidlaing
Any chance you will sign the CLA (and resolve the conflicts)? Otherwise we will close this PR within 2 weeks and open an issue to address your use case to allow other contributors to implement this feature.
Regards,
breml

[tsaarni]

It seems this PR is abandoned, so I created a new one for this feature #67.