The Kafka plugin in AliECS publishes updates messages about new states of environments and lists of environments in the RUNNING state. The messages are encoded with protobuf.
To enable the plugin, one should make sure that the following points are fullfiled.
- The consul instance includes coordinates to your kafka broker and enables the plugin.
Navigate to
o2/components/aliecs/ANY/any/settings
and make sure the following key value pairs are there:Please restart the AliECS core if you modify this file.kafkaEndpoint: "my-kafka-broker:9092" integrationPlugins: - kafka
- Plugin is enabled for the new environments. Make sure that there is a
true
value set in the consul instance at the patho2/runtime/aliecs/vars/kafka_enabled
. Alternatively, one can putkafka_enabled : true
in the Advanced configuration panel in the AliECS GUI.
As for today, AliECS publishes on the following types of topics:
aliecs.env_state.<state>
wherestate
can beSTANDBY
,DEPLOYED
,CONFIGURED
,RUNNING
,ERROR
,UNKNOWN
. For each topic, AliECS publishes aNewStateNotification
message when any environment reaches the corresponding state. TheUNKNOWN
state is usually published when an environment gets aDESTROY
request, but the plugin cannot know what will be the state after the transition.aliecs.env_leave_state.<state>
wherestate
can beSTANDBY
,DEPLOYED
,CONFIGURED
,RUNNING
,ERROR
. For each topic, AliECS publishes aNewStateNotification
message when any environment is about to leave the corresponding state.aliecs.env_list.<state>
wherestate
is onlyRUNNING
. Each time there is an environment state change, AliECS publishes anActiveRunsList
message which contains a list of all environments which are currently inRUNNING
state.
Messages are encoded with protobuf. Please use this proto file to generate code which deserializes the messages.
To get SOR and EOR notifications, please subscribe to the two corresponding topics:
aliecs.env_state.RUNNING
for Start of Runaliecs.env_leave_state.RUNNING
for End of Run
Both will provide NewStateNotification
messages encoded with protobuf. Please note that the EOR message will still contain the RUNNING state, because it is sent just before the transition starts.
One can use some Kafka command line tools to verify that a given setup works correctly. One should make sure to have Kafka installed on the machine used to run the tools.
To get a list of topics:
/opt/kafka/kafka_2.12-2.8.1/bin/kafka-topics.sh --bootstrap-server <kafka_broker_host>:<port> --list
To subscribe to a concrete topic:
/opt/kafka/kafka_2.12-2.8.1/bin/kafka-console-consumer.sh --bootstrap-server <kafka_broker_host>:<port> --topic aliecs.env_state.RUNNING
Please note that Kafka is distributes the messages in the push-pull mode by default. Thus, if you subscribe to messages with a debug tool, you might not see them in another application.