Skip to content

Commit

Permalink
Merge pull request #43 from lsst-sqre/afausti-patch-7
Browse files Browse the repository at this point in the history
Update connectors.rst
  • Loading branch information
afausti authored Oct 17, 2024
2 parents 3ef8705 + 6e9eac3 commit be3a7ab
Showing 1 changed file with 23 additions and 17 deletions.
40 changes: 23 additions & 17 deletions docs/developer-guide/connectors.rst
Original file line number Diff line number Diff line change
Expand Up @@ -86,41 +86,47 @@ For example, you might query the ``lsst.example.skyFluxMetric`` metric and group

See `InfluxDB schema design and data layout`_ for more insights on how to design tags.

Deployment and scaling
======================
Operations
==========

In Argo CD, sync the connector ConfigMap and the Deployment Kubernetes resources to deploy a connector.
Deployment
----------

To scale a connector horizontally, increase the ``kafkaConsumers.<connector name>.replicaCount`` value in the ``sasquatch/values-<environment>.yaml`` file.
To deploy a connector sync the connector ConfigMap and Deployment Kubernetes resources in Argo CD.

.. note::
List, stop and start connectors
-------------------------------

Note that scaling the connector horizontally only works if the Kafka topic has multiple partitions.
The number of topic partitions must be a multiple of the number of connector replicas.
For example if your topic was created with 8 partitions, you can scale the connector to 1, 2, 4, or 8 replicas.
To list the connectors in a given Sasquatch environment, run:

Operations
==========
.. code:: bash
To list the connectors deployed in a Sasquatch environment, run:
kubectl get deploy -l app.kubernetes.io/name=sasquatch-telegraf -n sasquatch
To view the view the logs of a single connector instance, run:

.. code:: bash
kubectl get deploy -l app.kubernetes.io/name=sasquatch-telegraf -n sasquatch
kubectl logs -l app.kubernetes.io/instance=sasquatch-telegraf-<connector-name> -n sasquatch
To view the logs of a connector or multiple connector instances run:
To stop the connectors you can scale the deployment replicas down to zero:

.. code:: bash
kubectl logs -l app.kubernetes.io/instance=sasquatch-telegraf-<connector-name> --tail=5 -n sasquatch
kubectl scale deploy -l app.kubernetes.io/name=sasquatch-telegraf --replicas=0 -n sasquatch
To stop a connector, run:
To start the connectors you can scale the deployment replicas back to one:

.. code:: bash
kubectl scale deploy/sasquatch-telegraf-<connector-name> --replicas=0 -n sasquatch
kubectl scale deploy -l app.kubernetes.io/name=sasquatch-telegraf --replicas=1 -n sasquatch
To permanently remove a connector set the ``kafkaConsumers.<connector name>.enabled`` key to ``false`` in the ``sasquatch/values-<environment>.yaml`` file and sync the connector ConfigMap and the Deployment Kubernetes resources in Argo CD.

Monitoring
----------

or set the ``kafkaConsumers.<connector name>.enabled`` key to ``false`` in the ``sasquatch/values-<environment>.yaml`` file and sync the connector ConfigMap and the Deployment Kubernetes resources in Argo CD.
Telegraf internal metrics are recorded under the ``telegraf`` database in Sasquatch and provide information about memory and buffer usage, throughput as well as read and write errors for each connector instance.


.. _InfluxDB v1 output: https://github.com/influxdata/telegraf/blob/master/plugins/outputs/influxdb/README.md
Expand Down

0 comments on commit be3a7ab

Please sign in to comment.