Backstop

Backstop is a simple endpoint for submitting metrics to Graphite. It accepts JSON data via HTTP POST and proxies the data to one or more Carbon/Graphite listeners.

My (kzarzycki) fork

This fork is modified to be able to accept metrics from Druid. If I had more time, I would document how to use it. In short, I've added quick & dirty '/druid' endpoint, that can accept metrics and alerts. Please look to backstop/blob/master/lib/backstop/web.rb for more details.

Usage

Collectd Metrics

Backstop supports submission of metrics via the Collectd write_http output plugin. A sample client configuration:

<Plugin write_http>
  <URL "https://backstop.example.com/collectd">
    Format "JSON"
    User ""
    Password ""
  </URL>
</Plugin>

GitHub Post-Receive Hooks

Backstop can receive commit data from GitHub post-receive webhooks. Your WebHook URL should consist of the Backstop service URL with the /github endpoint. For example, https://backstop.example.com/github.

All GitHub commit metrics contain the project name, branch information, author email and commit identifier, and are stored with a value of 1. These can then be visualized as annotation-style metrics using Graphite's drawAsInfinite() function. Sample metric:

github.project.refs.heads.master.bob-example-com.10af2cb02eadd4cb1a3e43aa9cae47ef2cd07016 1 1203116237

PagerDuty Incident Webhooks

Backstop can also receive PagerDuty incidents courtesy of Jesse Newland's pagerduty-incident-webhooks project. When deploying pagerduty-incident-webhooks make sure to set PAGERDUTY_WEBHOOK_ENDPOINT to your Backstop service URL with the /pagerduty endpoint. For example, https://backstop.example.com/pagerduty.

Metrics will be stored under the alerts prefix with a value of 1. These can then be visualized as annotation-style metrics using Graphite's drawAsInfinite() function. Sample metric:

alerts.nagios.web1.diskspace 1 1365206103

Custom Metrics

Use the /publish endpoint in conjunction with one of the approved PREFIXES for submitting metrics to Backstop. In most environments it makes sense to use distinct prefixes for normal (e.g. gauge, counters, etc) metrics vs annotation (event-style) metrics. PREFIXES is defined as a comma-delimited list of prefix strings. For example:

export PREFIXES='test,app1,app2'

Sending Metrics

Here is a basic example for posting an application metric to the custom prefix.

RestClient.post("https://backstop.example.com/publish/custom",
   [{:metric => key, :value => value, :measure_time => Time.now.to_i}].to_json)

Sending Annotations

Here is an example for posting a software release announcement to the note prefix.

RestClient.post("https://backstop.example.com/publish/note",
   [{:metric => "foobar.release", :value => "v214", :measure_time => Time.now.to_i}].to_json)

Using with Hosted Graphite

Graphite hosting service Hosted Graphite requires metrics to be submitted with an API key prepended to the metric. To use their service, just define the API_KEY environment variable.

Deployment

Backstop supports optional Basic Authentication through Rack::Auth::Basic. Simply set BACKSTOP_AUTH to your colon-delimited credentials (e.g. user:pass). Multiple credentials can be set, as long as they are comma-separated (e.g user1:pass1,user2:pass2).

The CARBON_URLS variable must be set to one or more valid destinations. Examples:

export CARBON_URLS="carbon://10.10.10.10:2003,carbon://10.10.20.10:2003"

Local

The following instructions assume a working Ruby installation with the bundler gem already installed on your system.

$ git clone https://github.com/obfuscurity/backstop.git
$ cd backstop
$ bundle install
$ export CARBON_URLS=...
$ export PREFIXES=...
$ export BACKSTOP_AUTH=... (optional)
$ foreman start

Heroku

$ heroku create
$ heroku config:add CARBON_URLS=...
$ heroku config:add PREFIXES=...
$ heroku config:add BACKSTOP_AUTH=... (optional)
$ git push heroku master

License

Backstop is distributed under a 3-clause BSD license.

Thanks

Thanks to Michael Gorsuch (@gorsuch) for his work on the collectd parser and the "Mitt" application that preceded Backstop.