Skip to content

Latest commit

 

History

History
104 lines (73 loc) · 5.07 KB

README.md

File metadata and controls

104 lines (73 loc) · 5.07 KB
Raw

Yōzefu

Minimum supported Rust version: 1.80.1 or plus Licence

Yōzefu is an interactive terminal user interface (TUI) application for exploring data of a kafka cluster. It is an alternative tool to AKHQ, redpanda console or the kafka plugin for JetBrains IDEs.

The tool offers the following features:

  • A real-time access to data published to topics.
  • A search query language inspired from SQL providing a fine-grained way filtering capabilities.
  • Ability to search kafka records across multiple topics.
  • Support for extending the search engine with user-defined filters written in WebAssembly (Extism).
  • The tool can be used as a terminal user interface or a CLI with the --headless flag.
  • One keystroke to export kafka records for further analysis.
  • Support for registering multiple kafka clusters, each with specific kafka consumer properties.

By default, the kafka consumer is configured with the property enable.auto.commit set to false, meaning no kafka consumer offset will be published to kafka.

Link to a demo video

Limitations

  • The tool is designed only to consume kafka records. There is no feature to produce records or manage a cluster.
  • Serialization formats such as json, xml or plain text are supported. Avro support is experimental for now. Protobuf is not supported.
  • The tool uses a ring buffer to store the last 500 kafka records.
  • There is probably room for improvement regarding the throughput (lot of clone() and deserialization).
  • Yozefu has been tested on MacOS Silicon but not on Windows or Linux. Feedback or contributions are welcome.

Getting started

Note

For a better visual experience, I invite you to install Powerline fonts.

cargo install yozefu

# By default, it starts the TUI. 
# The default registered cluster is localhost
yozf --cluster localhost

# You can also start the tool in headless mode.
# It prints the key of each kafka record matching the query in real time
yozf --cluster localhost               \
    --headless                         \
    --topics "public-french-addresses" \
    --format "json"                    \
    'from begin value.properties.type contains "street" and offset < 356_234 limit 10' \
  | jq '.key'


# Use the `configure` command to define new clusters
yozf configure

# You can create search filters
yozf create-filter --language rust key-ends-with

# And import them
yozf import-filter path/to/key-ends-with.wasm

You can also download pre-build binaries from the releases section. Attestions are available:

gh attestation verify --repo MAIF/yozefu <file-path of downloaded artifact>

Try it

Note

Docker is required to start a single node Kafka cluster on your machine. JBang is not required but recommended if you want to produce records with the schema registry.

# It clones this repository, starts a docker kafka node and produce some json records
curl -L "https://raw.githubusercontent.com/MAIF/yozefu/refs/heads/main/docs/try-it.sh" | bash

yozf -c localhost

Documentation