Skip to content

kube-logging/fluent-plugin-tag-normaliser

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

fluent-plugin-tag-normaliser

Tag-normaliser is a fluentd plugin to help re-tag logs with Kubernetes metadata coming from fluent-bit kubernetes filter.

Installation

RubyGems

$ gem install fluent-plugin-tag-normaliser

Bundler

Add following line to your Gemfile:

gem "fluent-plugin-tag-normaliser"

And then execute:

$ bundle

Configuration

You only need to specify the format option.

<match example.tag**>
  @type tag_normaliser
  format cluster1.${namespace_name}.${pod_name}.${labels.app}
</match>
Parameter Description Default
key_prefix Prefix used to access record attributes kubernetes
format Format to rewrite tag to. You can access record with ${record_name} expression. Nested attributes available via . separator: ${labels.app} ""
unknown Fallback value for missing record attribute "unknown"
sticky_tags Sticky tags will match only one record from an event stream. The same tag will be treated the same way true

Available fluent-bit provided kubernetes attributes

Parameter Description Example
pod_name Pod name understood-butterfly-nginx-logging-demo-7dcdcfdcd7-h7p9n
container_name Container name inside the Pod nginx-logging-demo
namespace_name Namespace name default
pod_id Kubernetes UUID for Pod 1f50d309-45a6-11e9-b795-025000000001
labels Kubernetes Pod labels. This is a nested map. You can access nested attributes via . {"app":"nginx-logging-demo", "pod-template-hash":"7dcdcfdcd7" }
host Node hostname the Pod runs on docker-desktop
docker_id Docker UUID of the container 3a38148aa37aa3...

Example

Configuration

<match example.tag**>
  @type tag_normaliser
  format cluster1.${namespace_name}.${pod_name}.${labels.app}
</match>

Input

tag = "kubernetes.var.log.containers.nginx-s6fvr_default_nginx-889831b33146bb9ec28a5700442b53c6cdf4c445fb5b71c0c48cfa27aaa00212.log"
{  
   "log":"10.1.0.1 - - [13/Mar/2019:15:42:31 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"kube-probe/1.13\" \"-\"\n",
   "kubernetes":{
      "pod_name":"understood-butterfly-nginx-logging-demo-7dcdcfdcd7-h7p9n",
      "namespace_name":"default",
      "pod_id":"1f50d309-45a6-11e9-b795-025000000001",
      "labels":{  
         "app":"nginx-logging-demo",
         "pod-template-hash":"7dcdcfdcd7",
         "release":"understood-butterfly"
      },
      "host":"docker-desktop",
      "container_name":"nginx-logging-demo",
      "docker_id":"3a38148aa37aa30e6e2df96af95cbda7a47b0428689bb4152413f4be25532fda"
   }
}

Output

tag = "cluster1.default.understood-butterfly-nginx-logging-demo-7dcdcfdcd7-h7p9n.nginx-logging-demo",

You can use the plugin without Kubernetes labels just set key_prefix to empty string.

Copyright

  • Copyright(c) 2019- Banzai Cloud
  • License
    • Apache License, Version 2.0

About

Fluent output plugin to transform tags based on record content

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages