add kafka exporter to quickstart (#481)

* add kafka exporter to quickstart * add kafka health dependency to kafka-exporter Co-authored-by: Jörg Zimmermann <[email protected]> --------- Co-authored-by: Jörg Zimmermann <[email protected]>
fkie-cad · Dec 15, 2023 · 8e8c741 · 8e8c741
1 parent 5fc46d7
commit 8e8c741
Show file tree

Hide file tree

Showing 8 changed files with 772 additions and 55 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -56,6 +56,7 @@
 * improve loading times for the rule tree by optimizing the rule segmentation and sorting
 * add support for python 3.12 and remove support for python 3.9
 * always check the existence of a field for negated key-value based lucene filter expressions
+* add kafka exporter to quickstart setup
 
 ### Bugfix
 

diff --git a/README.md b/README.md
@@ -430,17 +430,18 @@ The start up takes a few seconds to complete, but once everything is up
 and running it is possible to write JSON events into Kafka and read the processed events in
 Opensearch Dashboards. Following services are available after start up:
 
-| Service | Location | Credentials |
-|:----------|:----|:-----|
-| Kafka: | `localhost:9092` | |
-| Logprep metrics: | `localhost:8000` | |
-| Opensearch: | `localhost:9200` | |
+| Service                | Location | Credentials |
+|:-----------------------|:----|:-----|
+| Kafka:                 | `localhost:9092` | |
+| Kafka Exporter:        | `localhost:9308` | |
+| Logprep metrics:       | `localhost:8000` | |
+| Opensearch:            | `localhost:9200` | |
 | Opensearch Dashboards: | `localhost:5601` | |
-| Grafana Dashboards: | `localhost:3000` | admin:admin |
-| Prometheus: | `localhost:9090` | |
-| Nginx: | `localhost:8081` | user:password |
-| Keycloak: | `localhost:8080` | admin:admin |
-| Postgres: | `localhost:5432` | keycloak:bitnami |
+| Grafana Dashboards:    | `localhost:3000` | admin:admin |
+| Prometheus:            | `localhost:9090` | |
+| Nginx:                 | `localhost:8081` | user:password |
+| Keycloak:              | `localhost:8080` | admin:admin |
+| Postgres:              | `localhost:5432` | keycloak:bitnami |
 
 The example rules that are used in the docker instance of Logprep can be found 
 in `quickstart/exampledata/rules`.

diff --git a/quickstart/docker-compose.yml b/quickstart/docker-compose.yml
@@ -61,6 +61,23 @@ services:
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock
     command: sh -c "((sleep 15 && echo 'kafka up' && kafka-topics.sh --create --if-not-exists --bootstrap-server 127.0.0.1:9092 --replication-factor 1 --partitions 3 --topic consumer)&) && /opt/bitnami/scripts/kafka/run.sh"
+    healthcheck:
+      test:
+        [
+          "CMD-SHELL",
+          "kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --topic consumer --describe"
+        ]
+
+  kafka-exporter:
+    image: bitnami/kafka-exporter
+    container_name: kafkaexporter
+    command: [ "--kafka.server=127.0.0.1:9092" ]
+    network_mode: host
+    expose:
+      - 9308
+    depends_on:
+      kafka:
+        condition: service_healthy
 
   logprep:
     build: