AMQ Broker Operator Helm Enhancement

The aim is to automate the creation of the AMQ Broker Custom Resources and at the same time enhance the broker.xml with a configuration that is not yet available in the operator. The helm chart keeps in sync the custom resources and the custom broker.xml.

Prerequisites

AMQ Broker Operator must running in your target namespace.

Install AMQ Broker Helm Charts

helm upgrade --install mycluster amq-broker/

Test it

helm test mycluster

Custom Certificate Authority

The helm chart provides out of the box self signed certificate for each acceptor defined, but in case you want provide your custom certificate you need to create in advance the secret.

mkdir -p tls/{ca,broker}

Certificate Autority

This section is to simulate a private certificate authority.

Certificate Authority (CA)

openssl req -new -newkey rsa:2048 -x509 -keyout tls/ca/ca.key -out tls/ca/ca.crt -days 365 -subj "/CN=mycompany.com"

Create a Truststore

    keytool -import -storepass password -file tls/ca/ca.crt -alias mycompany.com -keystore tls/ca/client.ts

Broker Certificate

Broker Key

keytool -genkeypair -storepass password -keyalg RSA -keysize 2048 -dname "CN=mycluster-amq-broker-*-svc-rte-brk.apps-crc.testing" -ext san="dns:mycluster-amq-broker-all-0-svc-rte-brk.apps-crc.testing,dns:mycluster-amq-broker-all-0-svc.brk.svc" -alias broker -keystore tls/broker/broker.ks

Certificate Signing Request (CSR)

keytool -certreq -storepass password -keyalg rsa -alias broker -keystore tls/broker/broker.ks -file tls/broker/tls.csr

Certificate Authority Sign

openssl x509 -req -CA tls/ca/ca.crt -CAkey tls/ca/ca.key -in tls/broker/tls.csr -out tls/broker/tls.crt -days 365 -CAcreateserial

keytool -import -v -trustcacerts -alias root -file tls/ca/ca.crt -keystore tls/broker/broker.ks
keytool -import -v -trustcacerts -alias broker -file tls/broker/tls.crt -keystore tls/broker/broker.ks

Verify the chain

keytool -list -v -keystore tls/broker/broker.ks

Import to truststore

keytool -import -storepass password -file tls/broker/tls.crt -alias server -keystore tls/ca/client.ts

Create Secret

Note	Naming convention: `<CustomResourceName>-<AcceptorName>-secret`

oc create secret generic mycluster-amq-broker-all-secret \
  --from-file=tls/broker \
  --from-file=tls/ca/client.ts \
  --from-file=tls/ca/ca.crt \
  --from-literal=keyStorePassword=password \
  --from-literal=trustStorePassword=password

Important

Adopt with Helm

oc annotate secret/mycluster-amq-broker-all-secret meta.helm.sh/release-name=mycluster --overwrite
oc annotate secret/mycluster-amq-broker-all-secret meta.helm.sh/release-namespace=brk --overwrite
oc label secret/mycluster-amq-broker-all-secret app.kubernetes.io/managed-by=Helm --overwrite

Install with Custom Certificate

helm upgrade --install mycluster --set pki.self=false

Test External Client Communication

Extract Client TrustStore

oc extract secret/mycluster-amq-broker-all-secret --keys=client.ts
Extract AMQ Endpoint

export AMQ_ENDPOINT=$(oc get route mycluster-amq-broker-all-0-svc-rte --no-headers -o custom-columns=:.spec.host)

Launch Artemis Producer Command

./artemis producer --user amq --password amq --message-count 10 --url="tcp://${AMQ_ENDPOINT}:443?sslEnabled=true;trustStorePath=./client.ts;trustStorePassword=password"

Connection brokerURL = tcp://mycluster-amq-broker-all-0-svc-rte-brk.apps-crc.testing:443?sslEnabled=true;trustStorePath=./client.ts;trustStorePassword=password
Producer ActiveMQQueue[TEST], thread=0 Started to calculate elapsed time ...

Producer ActiveMQQueue[TEST], thread=0 Produced: 10 messages
Producer ActiveMQQueue[TEST], thread=0 Elapsed time in second : 0 s
Producer ActiveMQQueue[TEST], thread=0 Elapsed time in milli second : 79 milli seconds

Launch Artemis Consumer Command

./artemis consumer --user amq --password amq --message-count 10 --url="tcp://${AMQ_ENDPOINT}:443?sslEnabled=true;trustStorePath=./client.ts;trustStorePassword=password"

Connection brokerURL = tcp://mycluster-amq-broker-all-0-svc-rte-brk.apps-crc.testing:443?sslEnabled=true;trustStorePath=./client.ts;trustStorePassword=password
Consumer:: filter = null
Consumer ActiveMQQueue[TEST], thread=0 wait until 10 messages are consumed
Consumer ActiveMQQueue[TEST], thread=0 Consumed: 10 messages
Consumer ActiveMQQueue[TEST], thread=0 Elapsed time in second : 0 s
Consumer ActiveMQQueue[TEST], thread=0 Elapsed time in milli second : 17 milli seconds
Consumer ActiveMQQueue[TEST], thread=0 Consumed: 10 messages
Consumer ActiveMQQueue[TEST], thread=0 Consumer thread finished

Bridge Scenario

Create two OpenShift project.
```
oc new-project brk1
oc new-project brk2
```
Install AMQ Broker Operator for both the projects.

Create a NetworkPolicy to allow connection from brk1 to brk2.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mycluster1-amq-broker-all
  labels:
    app.kubernetes.io/name: amq-broker
    app.kubernetes.io/instance: mycluster2
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: amq-broker
      app.kubernetes.io/instance: mycluster2
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              app.kubernetes.io/instance: mycluster
      ports:
      - port: 61617
        protocol: TCP

Label brk1 namespace with app.kubernetes.io/instance: mycluster

oc label namespace brk1 app.kubernetes.io/instance=mycluster
Install AMQ Broker Helm Chart on brk2

helm upgrade --install mycluster2 amq-broker -n brk2 -f examples/bridge/values.brk2.yaml
Test the installation on brk2

helm test mycluster2 -n brk2

In case of SSL Connector to brk2, create a secret on brk1 to able to connect to brk2.

mkdir -p examples/bridge/brk2-certs

oc extract secret/mycluster2-amq-broker-all-secret --to=examples/bridge/brk2-certs -n brk2

oc create secret generic mycluster2-amq-broker-all-brk2-secret --from-file=examples/bridge/brk2-certs/ -n brk1

Install AMQ Broker Helm Chart on brk1

helm upgrade --install mycluster amq-broker -n brk1 -f examples/bridge/values.brk1.yaml
Test the installation on brk1

helm test mycluster -n brk1

Test the bridge connection

Note	Check Test External Client Communication to launch producer and consumer command.

Produce message on brk1

./artemis producer --user amq --password amq --message-count 10 --url="tcp://${AMQ_ENDPOINT}:443?sslEnabled=true;trustStorePath=./client.ts;trustStorePassword=password" --destination=bridge.test.address

Consume message on brk2

./artemis consumer --user amq --password amq --message-count 10 --url="tcp://${AMQ_ENDPOINT}:443?sslEnabled=true;trustStorePath=./client.ts;trustStorePassword=password" --destination=bridge.test.forwarding

Uninstall it

helm uninstall mycluster --no-hooks

Failed Status

In case of failed installation due to missing role rights, AMQ Broker operator, tests failed or others, please make sure to clean up all the pending resources.

oc delete all -lapp.kubernetes.io/name=amq-broker

Progress

Table 1. Progress

Task	Status
Add Address CRD logic	DONE
Patch AMQ Broker with custom dynamic configuration	DONE
Test Internal Connection Implementation	DONE
SSL Selfsigned Implementation	DONE
Test with External Client Implementation	DONE
Keystore and Truststore Password	DONE
SSL Custom CA Implementation	DONE #2
User Management	DONE #3
Diverts Implementation	DONE #1
Bridge Implementation	#1
Network Policy	#4
Make sure that the operator creates all the Addresses	TODO
High Availability and How scale down controller actives without the operator.	TODO
E2E Test- Migrate one standalone broker	TODO

Pseudo Code

Install Operator to specific namespace on the cluster

This can be provided by the cluster-admin as namespaced installation and should give the right role to manage the AMQ Broker CRD.

Note	You cannot create more than one broker deployment in a given OpenShift project by deploying multiple broker Custom Resource (CR) instances. However, when you have created a broker deployment in a project, you can deploy multiple CR instances for addresses. Reference.

[pre-install hook] Restore AMQ Broker Operator
[install/upgrade] Create kind: ActiveMQArtemis and ActiveMQArtemisAddress
[install/upgrade] Create Config Map with custom broker xml.
[post-install hook] Shutdown the AMQ Broker Operator

[post-install hook] Adopt the AMQ Broker resource to Helm:

annotations:
  meta.helm.sh/release-name: release-name
  meta.helm.sh/release-namespace: namespace-name
labels:
  app.kubernetes.io/managed-by: Helm

[post install] Adjust AMQ Broker Stateful set to use the custom broker xml.
1. Set BROKER_XML environment variable with your custom broker.xml.
[test] Verify the installation is correct.

Note	A *-hook install image requires oc client `quay.io/openshift/origin-cli:4.6` and running with edit role on the specific namespace.

Important

In AMQ Broker 7.7, if you want to configure any of the following items, you must add the appropriate configuration to the main CR instance before deploying the CR for the first time.
- Address settings
- The size of the Persistent Volume Claim (PVC) required by each broker in a deployment for persistent storage
- Limits and requests for memory and CPU for each broker in a deployment
During an active scaling event, any further changes that you apply are queued by the Operator and executed only when scaling is complete. For example, suppose that you scale the size of your deployment down from four brokers to one. Then, while scaledown is taking place, you also change the values of the broker administrator user name and password. In this case, the Operator queues the user name and password changes until the deployment is running with one active broker.
All CR changes – apart from changing the size of your deployment, or changing the value of the expose attribute for acceptors, connectors, or the console – cause existing brokers to be restarted. If you have multiple brokers in your deployment, only one broker restarts at a time.
To configure address and queue settings for broker deployments on OpenShift Container Platform, you add configuration to an addressSettings section of the main Custom Resource (CR) instance for the broker deployment. This contrasts with standalone deployments on Linux or Windows, for which you add configuration to an address-settings element in the broker.xml configuration file.
The format used for the names of configuration items differs between OpenShift Container Platform and standalone broker deployments. For OpenShift Container Platform deployments, configuration item names are in camel case, for example, defaultQueueRoutingType. By contrast, configuration item names for standalone deployments are in lower case and use a dash (-) separator, for example, default-queue-routing-type.
Addresses are created by the AMQ Broker Operator using Artemis Jolokia and MBean.

Reference

https://access.redhat.com/documentation/en-us/red_hat_amq/7.7/html-single/deploying_amq_broker_on_openshift/index#con-br-configuring-broker-certificate-for-hostname-verification_broker-ocp

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.adoc

README.adoc

AMQ Broker Operator Helm Enhancement

Prerequisites

Install AMQ Broker Helm Charts

Test it

Custom Certificate Authority

Certificate Autority

Broker Certificate

Create Secret

Install with Custom Certificate

Test External Client Communication

Bridge Scenario

Test the bridge connection

Uninstall it

Failed Status

Progress

Pseudo Code

Important

Reference

Files

README.adoc

Latest commit

History

README.adoc

File metadata and controls

AMQ Broker Operator Helm Enhancement

Prerequisites

Install AMQ Broker Helm Charts

Test it

Custom Certificate Authority

Certificate Autority

Broker Certificate

Create Secret

Install with Custom Certificate

Test External Client Communication

Bridge Scenario

Test the bridge connection

Uninstall it

Failed Status

Progress

Pseudo Code

Important

Reference