diff --git a/README.md b/README.md index 0b423d7a9..fd70445e5 100644 --- a/README.md +++ b/README.md @@ -394,11 +394,11 @@ Opensearch Dashboards. Following services are available after start up: The example rules that are used in the docker instance of Logprep can be found in `quickstart/exampledata/rules`. Example events that trigger for the example rules can be found in -`quickstart/exampledata/input_logdata/test_input.jsonl`. +`quickstart/exampledata/input_logdata/logclass/test_input.jsonl`. These events can be added to Kafka with the following command: ```bash -(docker exec -i kafka kafka-console-producer.sh --bootstrap-server 127.0.0.1:9092 --topic consumer) < exampledata/input_logdata/test_input.jsonl +(docker exec -i kafka kafka-console-producer.sh --bootstrap-server 127.0.0.1:9092 --topic consumer) < exampledata/input_logdata/logclass/test_input.jsonl ``` Once the events have been processed for the first time, the new indices *processed*, *sre* diff --git a/quickstart/exampledata/config/http_pipeline.yml b/quickstart/exampledata/config/http_pipeline.yml new file mode 100644 index 000000000..5be5501c4 --- /dev/null +++ b/quickstart/exampledata/config/http_pipeline.yml @@ -0,0 +1,25 @@ +version: 1 + +metrics: + enabled: true + port: 8000 + +input: + httpinput: + type: http_input + uvicorn_config: + host: 0.0.0.0 + port: 9000 + endpoints: + /lab/123/ABC/auditlog: jsonl +output: + kafka: + type: confluentkafka_output + topic: consumer + error_topic: errors + flush_timeout: 300 + send_timeout: 0 + kafka_config: + bootstrap.servers: 127.0.0.1:9092 + compression.type: gzip + statistics.interval.ms: "60000" diff --git a/quickstart/exampledata/input_logdata/logclass/config.yaml b/quickstart/exampledata/input_logdata/logclass/config.yaml new file mode 100644 index 000000000..aabcc476c --- /dev/null +++ b/quickstart/exampledata/input_logdata/logclass/config.yaml @@ -0,0 +1,7 @@ +target_path: /lab/123/ABC/auditlog +timestamps: + - key: TIMESTAMP_FIELD_1 + format: "%Y%m%d" + - key: TIMESTAMP_FIELD_1 + format: "%H%M%S" + time_shift: "+0200" diff --git a/quickstart/exampledata/input_logdata/test_input.jsonl b/quickstart/exampledata/input_logdata/logclass/test_input.jsonl similarity index 100% rename from quickstart/exampledata/input_logdata/test_input.jsonl rename to quickstart/exampledata/input_logdata/logclass/test_input.jsonl