Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable metadata fields for TCP output plugin #33

Open
cotej opened this issue Jun 2, 2020 · 1 comment
Open

Configurable metadata fields for TCP output plugin #33

cotej opened this issue Jun 2, 2020 · 1 comment

Comments

@cotej
Copy link

cotej commented Jun 2, 2020

This is a feature request to allow configurable default metadata fields (host, source, sourcetype, index) for the TCP output, as is currently supported by the HEC output.

I believe this should be possible by sending a header over the TCP socket as the first line of the payload, which should look like:

***SPLUNK*** <metadata field>=<string> <metadata field>=<string> ...

More info at: https://docs.splunk.com/Documentation/Splunk/8.0.4/Data/Assignmetadatatoeventsdynamically

@cotej
Copy link
Author

cotej commented Jun 2, 2020

I'm thinking something along the lines of this oughta do it. Seems to be working as expected from some basic testing.

--- a/lib/fluent/plugin/out_splunk_tcp.rb
+++ b/lib/fluent/plugin/out_splunk_tcp.rb
@@ -14,6 +14,11 @@ module Fluent
     config_param :host, :string
     config_param :port, :integer

+    config_param :default_host, :string, default: nil
+    config_param :default_source, :string, default: nil
+    config_param :default_sourcetype, :string, default: nil
+    config_param :default_index, :string, default: nil
+
     config_param :format, :string, default: 'raw'

     # for raw format
@@ -74,6 +79,20 @@ module Fluent
       else
         raise ConfigError, "invalid 'format' option: #{@format}"
       end
+
+      @headers = []
+      if @default_host
+        @headers << "host=#{@default_host}"
+      end
+      if @default_source
+        @headers << "source=#{@default_source}"
+      end
+      if @default_sourcetype
+        @headers << "sourcetype=#{@default_sourcetype}"
+      end
+      if @default_index
+        @headers << "index=#{@default_index}"
+      end
     end

     def multi_workers_ready?
@@ -92,6 +111,11 @@ module Fluent
       return if chunk.empty?

       payload = ''
+      unless @headers.empty?
+        payload << "***SPLUNK*** #{@headers.join(' ')}"
+        payload << @line_breaker
+      end
+
       chunk.msgpack_each do |time, record|
         event = @formatter.call(time, record)
         unless event.empty?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant