Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix[[docs]Update volumes in example yaml on getting-started-docker-compose.md #618

Closed
wants to merge 1 commit into from

Conversation

mikee805
Copy link

fix volumes configuration for docker compose.yaml

This PR fixes volumes config for docker compose
docker_compose_volumes_puslar_doc_update

  • [X ] doc
  • doc-required
  • doc-not-needed
  • doc-complete

fix volumes configuration
@github-actions github-actions bot added the doc Improvements or additions to documentation label Jun 25, 2023
@Anonymitaet Anonymitaet added this to the 3.1.0 milestone Jun 28, 2023
@Anonymitaet
Copy link
Member

@gaoran10
Could you please review this PR from a technical perspective? Thank you!

@gaoran10
Copy link
Contributor

Could you explain why need this change?

@mikee805
Copy link
Author

If you use the example as is the volumes will not mount

@gaoran10
Copy link
Contributor

I made a test with the current compose file, the cluster was generated successfully. Could you share the error logs you met?

version: '3'
networks:
  pulsar:
    driver: bridge
services:
  # Start zookeeper
  zookeeper:
    image: apachepulsar/pulsar:latest
    container_name: zookeeper
    restart: on-failure
    networks:
      - pulsar
    volumes:
      - ./data/zookeeper:/pulsar/data/zookeeper
    environment:
      - metadataStoreUrl=zk:zookeeper:2181
      - PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m
    command: >
      bash -c "bin/apply-config-from-env.py conf/zookeeper.conf && \
             bin/generate-zookeeper-config.sh conf/zookeeper.conf && \
             exec bin/pulsar zookeeper"
    healthcheck:
      test: ["CMD", "bin/pulsar-zookeeper-ruok.sh"]
      interval: 10s
      timeout: 5s
      retries: 30

  # Init cluster metadata
  pulsar-init:
    container_name: pulsar-init
    hostname: pulsar-init
    image: apachepulsar/pulsar:latest
    networks:
      - pulsar
    command: >
      bin/pulsar initialize-cluster-metadata \
               --cluster cluster-a \
               --zookeeper zookeeper:2181 \
               --configuration-store zookeeper:2181 \
               --web-service-url http://broker:8080 \
               --broker-service-url pulsar://broker:6650
    depends_on:
      zookeeper:
        condition: service_healthy

  # Start bookie
  bookie:
    image: apachepulsar/pulsar:latest
    container_name: bookie
    restart: on-failure
    networks:
      - pulsar
    environment:
      - clusterName=cluster-a
      - zkServers=zookeeper:2181
      - metadataServiceUri=metadata-store:zk:zookeeper:2181
      # otherwise every time we run docker compose uo or down we fail to start due to Cookie
      # See: https://github.com/apache/bookkeeper/blob/405e72acf42bb1104296447ea8840d805094c787/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java#L57-68
      - advertisedAddress=bookie
      - BOOKIE_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zookeeper:
        condition: service_healthy
      pulsar-init:
        condition: service_completed_successfully
    # Map the local directory to the container to avoid bookie startup failure due to insufficient container disks.
    volumes:
      - ./data/bookkeeper:/pulsar/data/bookkeeper
    command: bash -c "bin/apply-config-from-env.py conf/bookkeeper.conf && exec bin/pulsar bookie"

  # Start broker
  broker:
    image: apachepulsar/pulsar:latest
    container_name: broker
    hostname: broker
    restart: on-failure
    networks:
      - pulsar
    environment:
      - metadataStoreUrl=zk:zookeeper:2181
      - zookeeperServers=zookeeper:2181
      - clusterName=cluster-a
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - advertisedAddress=broker
      - advertisedListeners=external:pulsar://127.0.0.1:6650
      - PULSAR_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zookeeper:
        condition: service_healthy
      bookie:
        condition: service_started
    ports:
      - "6650:6650"
      - "8080:8080"
    command: bash -c "bin/apply-config-from-env.py conf/broker.conf && exec bin/pulsar broker"
➜  docker compose ps
NAME                COMMAND                  SERVICE             STATUS              PORTS
bookie              "bash -c 'bin/apply-…"   bookie              running
broker              "bash -c 'bin/apply-…"   broker              running             0.0.0.0:6650->6650/tcp, 0.0.0.0:8080->8080/tcp
pulsar-init         "bin/pulsar initiali…"   pulsar-init         exited (0)
zookeeper           "bash -c 'bin/apply-…"   zookeeper           running (healthy)

@mikee805
Copy link
Author

mikee805 commented Jul 4, 2023

I would have it up again I am on Linux if that makes a difference.

Do the proposed changes also work for you?

@gaoran10
Copy link
Contributor

gaoran10 commented Jul 5, 2023

Sorry, I didn't reproduce the problem, could you try to use my compose configuration and share the error logs you met?

@lhotari
Copy link
Member

lhotari commented Jan 24, 2024

Closing thi since #741 provides a solution to the problem. The data directories must be created and the ownership should be set to uid 10000 before starting the docker-compose environment.

@lhotari lhotari closed this Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants