A Puppet module for managing and configuring Logstash.
This module, "elastic/logstash" supports only Logstash 5.x. For earlier Logstash versions, support is provided by the legacy module, "elasticsearch/logstash".
- Puppet 3.8.6 or better.
- The stdlib Puppet library.
Optional:
- The apt (>= 2.0.0) Puppet library when using repo management on Debian/Ubuntu.
- The zypprepo Puppet library when using repo management on SLES/SuSE
This minimum viable configuration ensures that Logstash is installed, enabled, and running:
include logstash
# You must provide a valid pipeline configuration for the service to start.
logstash::configfile { 'my_ls_config':
content => template('path/to/config.file'),
}
class { 'logstash':
version => '5.0.2',
}
You may want to manage repositories manually. You can disable automatic repository management like this:
class { 'logstash':
manage_repo => false,
}
Rather than use your distribution's repository system, you can specify an explicit package to fetch and install.
class { 'logstash':
package_url => 'https://artifacts.elastic.co/downloads/logstash/logstash-5.1.1.rpm',
}
class { 'logstash':
package_url => 'puppet:///modules/my_module/logstash-5.1.1.rpm',
}
class { 'logstash':
package_url => 'file:///tmp/logstash-5.1.1.rpm',
}
class { 'logstash':
auto_upgrade => true,
}
class { 'logstash':
status => 'disabled',
}
Under normal circumstances, changing a configuration will trigger a restart of the service. This behaviour can be disabled:
class { 'logstash':
restart_on_change => false,
}
class { 'logstash':
ensure => 'absent',
}
Logstash uses several files to define settings for the service and associated Java runtime. The settings files can be configured with class parameters.
class { 'logstash':
settings => {
'pipeline.batch.size' => 25,
'pipeline.batch.delay' => 5,
}
}
class { 'logstash':
settings => {
'pipeline' => {
'batch' => {
'size' => 25,
'delay => 5,
}
}
}
}
class { 'logstash':
jvm_options => [
'-Xms1g',
'-Xmx1g',
]
}
class { 'logstash':
startup_options => {
'LS_NICE' => '10',
}
}
Pipeline configuration files can be declared with the logstash::configfile
type.
logstash::configfile { 'inputs':
content => template('path/to/input.conf.erb'),
}
or
logstash::configfile { 'filters':
source => 'puppet:///path/to/filter.conf',
}
For simple cases, it's possible to provide your Logstash config as an inline string:
logstash::configfile { 'basic_ls_config':
content => 'input { heartbeat {} } output { null {} }',
}
If you want to use hiera to specify your configs, include the following create_resources call in your node manifest or in manifests/site.pp:
$logstash_configs = hiera('logstash_configs', {})
create_resources('logstash::configfile', $logstash_configs)
...and then include the following config within the corresponding hiera file:
"logstash_configs": {
"config-name": {
"template": "logstash/config.file.erb",
}
}
Many plugins (notably Grok) use patterns. While many are included in Logstash already, additional site-specific patterns can be managed as well.
logstash::patternfile { 'extra_patterns':
source => 'puppet:///path/to/extra_pattern',
}
By default the resulting filename of the pattern will match that of the source. This can be over-ridden:
logstash::patternfile { 'extra_patterns_firewall':
source => 'puppet:///path/to/extra_patterns_firewall_v1',
filename => 'extra_patterns_firewall',
}
IMPORTANT NOTE: Using logstash::patternfile places new patterns in the correct directory, however, it does NOT cause the path to be included automatically for filters (example: grok filter). You will still need to include this path (by default, /etc/logstash/patterns/) explicitly in your configurations.
Example: If using 'grok' in one of your configurations, you must include the pattern path in each filter like this:
# Note: this example is Logstash configuration, not a Puppet resource.
# Logstash and Puppet look very similar!
grok {
patterns_dir => "/etc/logstash/patterns/"
...
}
logstash::plugin { 'logstash-input-beats': }
logstash::plugin { 'logstash-input-custom':
source => '/tmp/logstash-input-custom-0.1.0.gem',
}
logstash::plugin { 'logstash-filter-custom':
source => 'puppet:///modules/my_ls_module/logstash-filter-custom-0.1.0.gem',
}
logstash::plugin { 'x-pack':
source => 'https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-5.3.0.zip',
}
Need help? Join us in #logstash on Freenode IRC or on the https://discuss.elastic.co/c/logstash discussion forum.