Skip to content

Commit

Permalink
Add Fluentd instructions (contrib)
Browse files Browse the repository at this point in the history
  • Loading branch information
jpetazzo committed Dec 5, 2016
1 parent 16a4dac commit 6854698
Show file tree
Hide file tree
Showing 4 changed files with 77 additions and 0 deletions.
36 changes: 36 additions & 0 deletions efk/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Elasticsearch + Fluentd + Kibana

This is a variation on the classic "ELK" stack.

The [fluentd](fluentd/) subdirectory contains a Dockerfile to build
a fluentd image embarking a simple configuration file, accepting log
entries on port 24224 and storing them in Elasticsearch in a format
that Kibana can use.

You can also use a pre-built image, `jpetazzo/fluentd:v0.1`
(e.g. if you want to deploy on a cluster and don't want to deploy
your own registry).

Once this fluentd container is running, and assuming you expose
its port 24224/tcp somehow, you can send container logs to fluentd
by using Docker's fluentd logging driver.

You can bring up the whole stack with the associated Compoes file.
With Swarm mode, you can bring up the whole stack like this:

```bash
docker network create efk --driver overlay
docker service create --network efk \
--name elasticsearch elasticsearch:2
docker service create --network efk --publish 5601:5601 \
--name kibana kibana
docker service create --network efk --publish 24224:24224 \
--name fluentd jpetazzo/fluentd:v0.1
```

And then, from any node on your cluster, you can send logs to fluentd like this:

```bash
docker run --log-driver fluentd --log-opt fluentd-address=localhost:24224 \
alpine echo ohai there
```
24 changes: 24 additions & 0 deletions efk/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
version: "2"

services:
elasticsearch:
image: elasticsearch
# If you need to access ES directly, just uncomment those lines.
#ports:
# - "9200:9200"
# - "9300:9300"

fluentd:
#build: fluentd
image: jpetazzo/fluentd:v0.1
ports:
- "127.0.0.1:24224:24224"
depends_on:
- elasticsearch

kibana:
image: kibana
ports:
- "5601:5601"
environment:
ELASTICSEARCH_URL: http://elasticsearch:9200
5 changes: 5 additions & 0 deletions efk/fluentd/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
FROM ruby
RUN gem install fluentd
RUN gem install fluent-plugin-elasticsearch
COPY fluentd.conf /fluentd.conf
CMD ["fluentd", "-c", "/fluentd.conf"]
12 changes: 12 additions & 0 deletions efk/fluentd/fluentd.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<source>
@type forward
port 24224
bind 0.0.0.0
</source>

<match **>
@type elasticsearch
host elasticsearch
logstash_format true
flush_interval 1
</match>

0 comments on commit 6854698

Please sign in to comment.