Skip to content
This repository has been archived by the owner on Jun 16, 2023. It is now read-only.

Operation Experience

Jump to bottom
BlueSkyChina edited this page Oct 8, 2014 · 5 revisions

Operation Experience

  1. Supervisor or nimbus start in daemon mode , Avoid terminal exit, sending a signal to jstorm, leading to unexpectedly quit jstorm.

    $: nohup jstorm supervisor >/dev/null 2>&1 &

  2. It is recommended to use the admin user starts all programs, especially not use the root user

  3. In the installation directory, it is recommended to use jstorm-current link, such as the currently used version is jstorm 0.9.4, then create a link to jstorm-0.9.4, when after the upgrade, simply jstorm-current link to the new jstorm version. $: ln -s jstorm-0.9.4 jstorm-current

  4. Recommended setting JStorm's local data and log directory to a common directory, such as "/home/admin/jstorm_data" and "/home/admin/logs", don't configure to $JSTORM_HOME/data and $JSTORM_HOME/logs, because when you upgrade, replace the entire directory, easy to lose all local data and logs.

  5. JStorm support environment variables JSTORM CONF_DIR, when this variable is set, Jstorm reads storm.yaml files from this directory, it is recommended to set this variable, the directory specified by the variable to store configuration files storm.yaml, after each upgrade , you can simply replace the directory.

  6. Recommends no more than one month, forced restart the supervisor, because the supervisor is a daemon process, constantly create child processes, when used for too long, the file will be very much open handles, resulting in launch worker is slow, Therefore, recommend every other week, forced to restart supervisor once.

  7. At the time of kill or rebalance a topology, it is best to deactivate this topology, then wait MSG_TIMEOUT time, and then kill or rebalance

  8. JStorm web ui recommend using apache tomcat 7.x, the default port is 8080, if you need to redirect port 80 to 8080, you can execute commands with root $:sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

  9. Jvm GC need to use CMS GC mode, JStorm's default setting, use the Storm need similar settings. worker.childopts: "-Xms1g -Xmx1g -Xmn378m -XX:SurvivorRatio=2 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=65"

  10. For some important applications can be grouped large clusters, modify "storm.zookeeper.root" and "nimbus.host" profile

  11. Recommend using 2.1.7 version Zeromq

    • A 64-bit Java will need to use the 64 zeromq
    • In a 64-bit OS using 32-bit Java, compile zeromq flag - m32 increase

  12. For applications using ZK more frequently, you need to the zk of JStorm and application isolation, will not mix use

  13. the nimbus node does not recommend running supervisor, and suggested that the nimbus placed on the machine where the run ZK

  14. Recommended slot for "CPU core -1", assuming 24-core CPU, the slot is 23

  15. Configuration cronjob, regularly check nimbus and supervisor, once the process of dying, automatic restart

  16. Zookeeper's maxClientCnxns=500

  17. Linux exists the external connection port limit, TCP client initiates a connection outside reached about 28,000, began to throw a lot of exceptions, you need to modify the external connection port restrictions

$:echo "10000 65535" > /proc/sys/net/ipv4/ip_local_port_range

Clone this wiki locally