Configuration

The Burrow configuration file is an INI-style config, specifically of the flavor understood by the gcfg module for Golang. The configuration is organized into several subheadings. There is an example configuration file in the "config" directory of the Burrow distribution.

General

The [general] configuration heading covers the location of log and PID files, as well as some shared configuration for Kafka clusters.

[general]
logdir=log
logconfig=config/logging.cfg
pidfile=burrow.pid
client-id=burrow-lagchecker
group-blacklist=^(console-consumer-|python-kafka-consumer-).*$

Key	Value Type	Required	Default Value	Description
logdir	directory	no	(current working dir)	A path to the directory that will contain the log and PID files. The path is relative to the directory in which Burrow is run.
logconfig	path to file	no	(none)	A path to the file specifying a custom format to use for logging. For details on the format of this file, refer to the seelog wiki
pidfile	filename	no	burrow.pid	The filename to store the PID of the running process in. This file will be placed in the logdir directory
client-id	string	no	burrow-client	The client ID string to provide to Kafka when consuming. This follows the same rules as topic names (alphanumeric, plus dash and underscore)
group-blacklist	regular expression	no	(none)	If specified, consumer group names that match this RE are ignored

Zookeeper

The [zookeeper] heading specifies a Zookeeper ensemble to use in order to provide locking between multiple copies of Burrow. This allows you to run multiple copies of Burrow with only one being allowed to execute the notifiers at any time.

[zookeeper]
hostname=zkhost01.example.com
hostname=zkhost02.example.com
hostname=zkhost03.example.com
port=2181
timeout=6
lock-path=/burrow/notifier

Key	Value Type	Required	Default Value	Description
hostname	hostname OR hostname:port	yes	(none)	The fully qualified domain name of a Zookeeper node. Multiple hostname entries can be provided for redundancy.
port	integer	no	2181	The client port number for the Zookeeper ensemble (used when not specified per-host).
timeout	integer	no	6	The connection timeout for Zookeeper sessions, in seconds. This value is also used for the Kafka cluster Zookeeper connections.
lock-path	znode	no	/burrow/notifier	A full path to the znode to use for locking between Burrow instances. The path will be created if it doesn't already exist.

Kafka

The [kafka] heading specifies a single Kafka cluster for Burrow to connect to and monitor. The subheading ("local" in the example below) is the name of the Kafka cluster that is used to identify it in HTTP requests and notifications. Multiple [kafka] headings can be specified to provide for monitoring many Kafka clusters with one Burrow instance.

[kafka "local"]
broker=kafka01.example.com
broker=kafka02.example.com
broker=kafka03.example.com
broker=kafka04.example.com
broker=kafka05.example.com
broker-port=10251
zookeeper=zkhost01.example.com
zookeeper=zkhost02.example.com
zookeeper=zkhost03.example.com
zookeeper-port=2181
zookeeper-path=/kafka-cluster
zookeeper-offsets=true
offsets-topic=__consumer_offsets

Key	Value Type	Required	Default Value	Description
broker	hostname OR hostname:port	yes	(none)	The fully qualified domain name of a Kafka broker. Multiple broker entries can be provided for redundancy.
broker-port	integer	no	9092	The client port number for the Kafka cluster (used when not specified per-host)
zookeeper	hostname OR hostname:port	yes	(none)	The fully qualified domain name of a Zookeeper node. Multiple zookeeper entries can be provided for redundancy.
zookeeper-port	integer	no	2181	The client port number for the Zookeeper ensemble (used when not specified per-host)
zookeeper-path	znode	yes	(none)	The full path to the znode that is the root for the Kafka cluster. If Kafka is configured with a chroot path for Zookeeper, "/" will specify the root znode.
zookeeper-offsets	boolean	no	false	Selects whether or not Zookeeper committed offsets for this Kafka cluster are collected
offsets-topic	string	no	__consumer_offsets	The name of the consumer offsets topic. This should not need to be specified at all.

Storm

The [storm] heading specifies a single Storm cluster for Burrow to monitor through Zookeeper offsets. The subheading ("local" in the example below) is the name of the Kafka cluster that this Storm cluster maps to, and is used to identify it in HTTP requests and notifications. Multiple [storm] headings can be specified to provide for monitoring many Storm clusters with one Burrow instance.

[storm "local"]
zookeeper=zkhost01.example.com
zookeeper=zkhost02.example.com
zookeeper=zkhost03.example.com
zookeeper-port=2181
zookeeper-path=/storm-cluster

Key	Value Type	Required	Default Value	Description
zookeeper	hostname OR hostname:port	yes	(none)	The fully qualified domain name of a Zookeeper node. Multiple zookeeper entries can be provided for redundancy.
zookeeper-port	integer	no	2181	The client port number for the Zookeeper ensemble (used when not specified per-host)
zookeeper-path	znode	yes	(none)	The full path to the znode that is the root for the Storm cluster. If Storm is configured with a chroot path for Zookeeper, "/" will specify the root znode. Multiple paths can be provided for when consumers write their offsets to different locations.

Notify center

The [notify] heading holds configuration regarding the notification center.

[notify]
interval=10

Key	Value Type	Required	Default Value	Description
interval	integer	no	10	The interval to check for consumer group status

Tickers

The [tickers] heading holds configurations regarding how often to execute certain actions.

[tickers]
broker-offsets=60

Key	Value Type	Required	Default Value	Description
broker-offsets	integer	no	60	How often to refresh the broker HEAD (end of log) offsets for all partitions, in seconds.

Lagcheck

The [lagcheck] heading holds configurations regarding the internal lag checking algorithm. For more information about how these configurations are used, please see the Consumer Lag Evaluation Rules page.

[lagcheck]
intervals=10
expire-group=604800
min-distance=1
zookeeper-interval=60
zk-group-refresh=300

Key	Value Type	Required	Default Value	Description
intervals	integer	no	10	How many consumer offsets to store for a partition. This determines the window over which to evaluate consumer lag.
expire-group	integer	no	604800	How long to wait, in seconds, after a consumer has stopped committing offsets to remove it from Burrow's monitoring.
min-distance	integer	no	1	The minimum interval, in seconds, allowed between consumer offset commits. Commits under this interval are ignored.
zookeeper-interval	integer	no	60	How frequently, in seconds, to scan Zookeeper committed offsets for clusters that are configured for ZK offsets.
zk-group-refresh	integer	no	300	How frequently, in seconds, to refresh the list of consumer groups for which Zookeeper offsets are collected.

HTTPServer

The [httpserver] heading configures the HTTP endpoint in Burrow.

[httpserver]
server=on
port=8000
; Alternately, use listen
; listen=host:port
; listen=host2:port2

Key	Value Type	Required	Default Value	Description
server	boolean	no	false	If set to true (or a value equivalent to true, such as "on"), the HTTP server is enabled.
port	integer	no	(none)	The TCP port number to run the HTTP server on. Either port option or listen option is required if the HTTP server is enabled, but specifying both is not allowed.
listen	host:port	no	(none)	Listen address to use. May be specified multiple times to listen on multiple addresses. Either port option or listen option is required if the HTTP server is enabled, but specifying both is not allowed.

Email Notifier

Configuration for the email notifier is covered on the Email Notifier page.

HTTP Notifier

Configuration for the HTTP notifier is covered on the HTTP Notifier page.