-
Notifications
You must be signed in to change notification settings - Fork 43
Mission: Circuit Breaker
| ID | Short Name |
|---|---|
|
|
The purpose of this Mission is to demonstrate how the circuit breaker works when a service fails or becomes unavailable.
To demonstrate this behavior, we will configure a /api/greeting HTTP endpoint which is called by the client to get a personalized greeting. This greeting endpoint then issues a request to a name service, which is hidden behind the /api/name HTTP endpoint. If this name service is fully functional, we get back service’s “World” response (or some other name), otherwise circuit breaker should get the name from the fallback option (a defined fallback method, e.g. could be simple “Fallback”).
To have the name service fail or become unavailable, we have two options. We can either kill the pod with the name service, or a more graceful one, to have unavailability programmed into the service itself — to toggle / switch a name service to throw an exception / error (e.g. return a HTTP 500 response code).
This should transition the circuit breaker to go to “open” state. We can then toggle back the name service, to again work properly, hence transition the circuit breaker back to “closed” state.
We should also be able to query the current state of the name service’s circuit breaker state — via another /api/cb-state HTTP endpoint or some other technology (e.g. web socket). This can / should be nicely visualised in the entry HTML page (index.html).
Implementing a Circuit Breaker and fallback mechanism for Microservice-bsed
The Circuit Breaker pattern provides a generic mechanism for quickly and easily reporting whether or not a service is functioning properly. This allows a service to report information about a failure and decide whether to quickly fail or simply degrade gracefully instead of failing.
Modern cloud applications are often comprised of many smaller services which interact with each other in a many-to-many manner. Understanding when a service is failing or unavailable is critically important, which is the main reason for the Health Check pattern. In some cases though, you may want to provide an alternative response if a service interacts with a failing or unavailable service. For example, if service1 interacts with service2 and service2 begins to fail or become unavailable due to high load, simply detecting the failure and restarting service2 may not alleviate the issue. Instead, you may want to signal service1 to reduce the amount of load it puts on service2 until service2 returns to a normal state.
The runtime (Spring Boot, Swarm, Vert.x) provides the code or the jar file containing the circuit breaker.
Success scenario
The use case starts when the application has been deployed into OpenShift The user can access the application using a web page provided by the application where the following scenario will be proposed : Call the api/cb-state endpoint using the button “cb-state” to check that the status of the circuit breaker returned as plain text is “closed”. Or if you’re using web sockets, the label should display “closed”. 3.a. (if hystrix/turbine dashboard are deployed) Get the hystrix dashboard URL using the command (“oc get route/hystrix-dashboard or minishift openshift service hystrix-dashboard -n your_namespace --url”) and open it with your browser. Click on the button (“monitor stream”). You will be redirected to the turbine server. Under the Circuit section of the screen, verify that the C-B is well monitored and is current status is “closed”. Click on the “greeting service” button to call api/greeting Verify that a JSON response message is received {"content": "Hello, World!"} Now let’s check a fallback response. We first need to toggle the name service to fail - click on the “toggle” button to call api/state endpoint, its label should change to “Fail” (now name service should return 500 HTTP response code when the CB will call it) Again click on the “greeting service” button to call api/greeting Verify that a JSON response message now received is {"content": "Hello, Fallback!"} Call the api/cb-state endpoint using the button “cb-state” to check that the status of the circuit breaker returned as plain text is “open” 10.a. (if hystrix/turbine dashboard are deployed) Open the turbine dashboard. Under the Circuit section of the screen, verify that the C-B status is “open”. Let’s now toggle the name service back to OK - click on the “toggle” button (to call api/state endpoint), its label should change to “OK” (and name service should stop returning 500 HTTP response code, it should be back to 200 HTTP response code) Click on the “greeting service” button a few times to call api/greeting, until C-B timeout expires (moving from open to half-open and if it succeeds to closed). In parallel, click also on the “cb-state” button to check the status returned is “closed” again. Or if you’re using web sockets, the label should again display “closed”. Verify that a JSON response message received is {"content": "Hello, World!"}
The use case will consist of :
Develop a HTTP application which uses 2 pods Maven project is designed as 2 maven modules; greeting_service and name_service. No parent pom.xml file is created for the 2 modules Each pod is created from the openshift resources generated by f-m-p executed within each Maven module First “name-service” pod contains a /api/name endpoint, which will be used to provide the name (in a JSON content) and a /api/state endpoint, to toggle the name service’s functionality. Second “greeting-service” pod contains a /api/greeting endpoint, which will return a json greeting message, getting the name from the name endpoint and a /api/cb-state endpoint, to get current name service’s circuit breaker state. Greeting endpoint should access the name endpoint via circuit breaker. The Hystrix Dashboard and Turbine server could be deployed to visualize the status about the C-B change. We should find a way when we will deploy this mission on OpenShift Platform to check if the namespace of the user still has enough resources (CPU/memory) to deploy Hystrix/Turbine. If this is not the case, then they will not be installed and we will mention within the README/Doc the scenario to be followed to check the state of the Circuit-Breaker. We could use as input what this command returns (“oc describe limits <LIMIT_NAME>”) to calculate if they could be deployed.
During nominal work, a curl or http request issued against the following service $protocol://$hostname:$port/api/greeting returns { "content": "Hello, World!"} If we toggle the name endpoint / service — issuing GET request: $protocol://$hostname:$port/api/state?state=fail (this should make the name service return a 500 HTTP response code) we should get a fallback - a curl or http request issued against the following service $protocol://$hostname:$port/api/greeting returns { "content": "Hello, Fallback!"} When we toggle back the name service — issuing GET request with: $protocol://$hostname:$port/api/state?state=ok The greeting endpoint $protocol://$hostname:$port/api/greeting should return { "content": "Hello, World!"} During this state transitions, we can issue http request against circuit breaker state endpoint / web socket and should receive current state: $protocol://$hostname:$port/api/cb-state should return one of these responses: {state:"open”} or {state:"closed”}
Coordination Owner |
☑ |
|
Product Management |
☑ |
|
DevExp |
☑ |
|
Vert.x |
☑ |
|
WildFly Swarm |
☑ |
|
Spring Boot |
☑ |
|
QE |
☑ |
|
Docs |
☑ |
|
Architect |
☑ |