Add kafka docker volumes to scaled kafka broker

[isuru-sam]

Hi,
I want to add persistent volumes to the entire cluster with 3 broker nodes as below.
docker-compose up -d
docker-compose scale kafka=3

If I simply add volume definition as below to the docker compose yml file will the volume be shared by all brokers without conflict or do i need to up brokers independently using docker-compose-single-broker.yml with different volume definitions?

volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /home/srv/kafka:/kafka/kafka-logs-1
Thanks
Isuru

[sscaling]

It would depend on your BROKER_ID / LOGS_DIR configuration. They'd need to be structured so they wrote broker data as distinct sub directories.

Typically, with docker-compose, I've seen people use multiple broker definitions instead of the scale command. e.g.

services:
  zookeeper:
     ...
  kafka_one:
    environment:
      KAFKA_BROKER_ID: 1
      LOGS_DIR: /kafka/kafka-logs-1
    volumes:
      - /home/srv/kafka/kafka-logs-1:/kafka/kafka-logs-1
  kafka_two:
    environment:
      KAFKA_BROKER_ID: 2
      LOGS_DIR: /kafka/kafka-logs-2
    volumes:
      - /home/srv/kafka/kafka-logs-2:/kafka/kafka-logs-2

etc...

[SimplyAmuthan]

If I want to scale Kafka docker volumes? how can I do that, I purposely don't want to use multiple broker definitions in my docker-compose file. I would like to scale like
docker-compose up --scale kafka=3