replace (brittle) promtail-based Docker logging with new loki-docker-…

…driver

README.md

@@ -4,20 +4,32 @@ Docker compose file for Grafana Loki, like Prometheus but for logs. Requires Doc
 
 ## INSTALLATION
 
-On every one of your Docker nodes, create a directory and cron job that will populate container names:
+On every Docker swarm node, first install Loki's Docker log driver plugin (it's important to install the plugin before modifying /etc/docker/daemon.json)
 
 ```
-mkdir -p /etc/promtail/targets
-crontab -e
+docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
 ```
 
-and add the following line to your root cron (or cron of a user that can execute /bin/docker):
+Next, add the following (at least the log-driver and log-opts below) to the file /etc/docker/daemon.json on each Docker swarm node:
 
 ```
-* * * * *  docker ps --no-trunc --format '- targets: ["{{.ID}}"]\n  labels:\n    container_name: "{{.Names}}"' > /etc/promtail/targets/promtail-targets.yaml
+{
+    "debug" : false,
+    "log-driver": "loki",
+    "log-opts": {
+        "loki-url": "http://SWARM-FQDN-OR-IP:3100/api/prom/push",
+        "loki-batch-size": "400"
+    }
+}
 ```
 
-Non-[swarmstack](https://github.com/swarmstack/swarmstack) users will need to edit promtail-docker-config.yaml and change the clients url from http://loki:3100/.. to the IP or DNS of one of your Docker nodes.
+Finally, restart the Docker daemon on each swarm node (draining nodes first if necessary) to pick up the logging change:
+
+```
+docker node update --availbility drain node1.fqdn
+systemctl restart docker
+docker node update --availbility active node1.fqdn
+```
 
 ## USAGE
 

docker-compose-swarmstack.yaml → docker-compose-swarmstack.yml

@@ -15,8 +15,6 @@ volumes:
 configs:
   loki_config:
     file: ./loki-local-config.yaml
-  promtail_config:
-    file: ./promtail-docker-config.yaml
 
 services:
   loki:
@@ -35,21 +33,7 @@ services:
         constraints:
           - node.Labels.storagegroup==RED
       replicas: 1
+    ports:
+      - "3100:3100"
     volumes:
       - loki-data:/data/loki
-
-  promtail:
-    image: grafana/promtail:v0.1.0
-    command: -config.file=/etc/promtail/promtail-docker-config.yaml
-    configs:
-      - source: promtail_config
-        target: /etc/promtail/promtail-docker-config.yaml
-    deploy:
-      labels:
-        prometheus.enable: "true"
-        prometheus.port: "9080"
-        prometheus.path: "/metrics"
-      mode: global
-    volumes:
-      - /var/lib/docker/containers:/var/lib/docker/containers:ro
-      - /etc/promtail/targets:/etc/promtail/targets:ro

docker-compose.yml

@@ -0,0 +1,20 @@
+version: "3.4"
+
+configs:
+  loki_config:
+    file: ./loki-local-config.yaml
+
+services:
+  loki:
+    image: grafana/loki:v0.1.0
+    command: -config.file=/etc/loki/loki-local-config.yaml
+    configs:
+      - source: loki_config
+        target: /etc/loki/loki-local-config.yaml
+    deploy:
+      mode: replicated
+      replicas: 1
+    ports:
+      - "3100:3100"
+    volumes:
+      - ./local_bind_volume_dir:/data/loki

up

@@ -1,3 +1,3 @@
 #!/bin/sh
 
-docker stack deploy -c docker-compose.yaml loki
+docker stack deploy -c docker-compose.yml loki

up-swarmstack

@@ -1,3 +1,3 @@
 #!/bin/sh
 
-docker stack deploy -c docker-compose-swarmstack.yaml loki
+docker stack deploy -c docker-compose-swarmstack.yml loki