Skip to content

Configurate Storm Cluster

Kaain edited this page Nov 27, 2012 · 3 revisions

Open Ports needed:

Port Service
22 SSH
2181 Zookeeper connect
2888 and 3888 If Zookeeper in Cluster
6627 thrift
6700 - 6703 Connection to Supervisors
8080 HTTP for UI

Server

  • 1 EC2 Redis Server
  • 1 EC2 Zookeeper Server
  • 1 EC2 Storm Nimbus
  • 1 EC2 Storm Supervisor

Configuration

storm/conf/storm.yaml (Supervisor)

storm.zookeeper.servers:
  - "ec2 for zookeeper"
storm.zookeeper.port: 2181
storm.local.dir: "/tmp/storm"
nimbus.host: "ec2 for nimbus"
supervisor.slots.ports:
  - 6700
  - 6701
  - 6702
  - 6703

supervisor.slots.ports the number of superversors. 4 Ports = 4 Supervisors

storm/conf/storm.yaml (Nimbus)

storm.zookeeper.servers:
  - "ec2 for zookeeper"
storm.zookeeper.port: 2181
storm.local.dir: "/tmp/storm"
supervisor.slots.ports:
  - 6700
  - 6701
  - 6702
  - 6703

supervisor.slots.ports if you want supervisors on the nimbus machine

zookeeper-3.4.5/conf/zoo.cfg (Zookeeper)

tickTime=2000
dataDir=/var/zookeeper/
clientPort=2181

This is for a single Zookeeper node. See ZooKeeper Administrator's Guide for MultiServerSetup.