Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(helm): add OpenSearch Operator Helm chart #513

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions opensearch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: OpenSearch
---

## OpenSearch Plugin

The **OpenSearch** plugin sets up an OpenSearch environment using the **OpenSearch Operator**, automating deployment, provisioning, management, and orchestration of OpenSearch clusters and dashboards. It functions as the backend for logs gathered by collectors such as OpenTelemetry collectors, enabling storage and visualization of logs for Greenhouse-onboarded Kubernetes clusters.

The main terminologies used in this document can be found in [core-concepts](https://cloudoperators.github.io/greenhouse/docs/getting-started/core-concepts).

## Overview

OpenSearch is a distributed search and analytics engine designed for real-time log and event data analysis.
The **OpenSearch Operator** simplifies the management of OpenSearch clusters by providing declarative APIs for configuration and scaling.

### Components included in this Plugin:

- [OpenSearch Operator](https://github.com/opensearch-project/opensearch-k8s-operator)
- OpenSearch Cluster Management
- OpenSearch Dashboards Deployment
- OpenSearch Index Management
- OpenSearch Security Configuration

## Architecture

![OpenSearch Architecture](img/opensearch-arch.png)

The OpenSearch Operator automates the management of OpenSearch clusters within a Kubernetes environment. The architecture consists of:

- **OpenSearchCluster CRD**: Defines the structure and configuration of OpenSearch clusters, including node roles, scaling policies, and version management.
- **OpenSearchDashboards CRD**: Manages OpenSearch Dashboards deployments, ensuring high availability and automatic upgrades.
- **OpenSearchISMPolicy CRD**: Implements index lifecycle management, defining policies for retention, rollover, and deletion.
- **OpenSearchIndexTemplate CRD**: Enables the definition of index mappings, settings, and template structures.
- **Security Configuration via OpenSearchRole and OpenSearchUser**: Manages authentication and authorization for OpenSearch users and roles.

## Note

More configurations will be added over time, and contributions of custom configurations are highly appreciated.
If you discover bugs or want to add functionality to the plugin, feel free to create a pull request.

## Quick Start

This guide provides a quick and straightforward way to use **OpenSearch** as a Greenhouse Plugin on your Kubernetes cluster.

### Prerequisites

- A running and Greenhouse-onboarded Kubernetes cluster. If you don't have one, follow the [Cluster onboarding](https://cloudoperators.github.io/greenhouse/docs/user-guides/cluster/onboarding) guide.
- The OpenSearch Operator installed via Helm or Kubernetes manifests.
- An OpenTelemetry or similar log ingestion pipeline configured to send logs to OpenSearch.

### Installation

#### Install via Greenhouse

1. Navigate to the **Greenhouse Dashboard**.
2. Select the **OpenSearch** plugin from the catalog.
3. Specify the target cluster and configuration options.

## Configuration

| Name | Description | Type | Required |
|-------------------------------------------|----------------------------------------------------------|--------|----------|
| `openSearch.cluster.name` | Name of the OpenSearch cluster | string | `true` |
| `openSearch.cluster.version` | OpenSearch version to deploy | string | `true` |
| `openSearch.cluster.replicas.master` | Number of master nodes | int | `false` |
| `openSearch.cluster.replicas.data` | Number of data nodes | int | `false` |
| `openSearch.cluster.replicas.dashboards` | Number of OpenSearch Dashboards nodes | int | `false` |
| `openSearch.security.username` | Admin username for OpenSearch authentication | secret | `true` |
| `openSearch.security.password` | Admin password for OpenSearch authentication | secret | `true` |
| `openSearch.security.roles` | User role mappings | map | `false` |
| `openSearch.index.ismPolicy.enabled` | Enable index lifecycle management | bool | `false` |
| `openSearch.index.ismPolicy.policy` | Custom ISM policy for index management | map | `false` |
| `openSearch.index.template.enabled` | Enable index templates | bool | `false` |
| `openSearch.index.template.definition` | Custom index template definition | map | `false` |
| `openSearch.monitoring.prometheus.enabled` | Enable Prometheus monitoring | bool | `false` |
| `openSearch.monitoring.serviceMonitor` | Deploy a ServiceMonitor for Prometheus integration | bool | `false` |

## Usage

Once deployed, OpenSearch can be accessed via OpenSearch Dashboards.

```sh
kubectl port-forward svc/opensearch-dashboards 5601:5601
```

Visit `http://localhost:5601` in your browser and log in using the configured credentials.

## Conclusion

This guide ensures that OpenSearch is fully integrated into the Greenhouse ecosystem, providing scalable log management and visualization.
Additional custom configurations can be introduced to meet specific operational needs.

For troubleshooting and further details, check out the [OpenSearch documentation](https://opensearch.org/docs/).
9 changes: 9 additions & 0 deletions opensearch/chart/Chart.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
dependencies:
- name: opensearch-operator
repository: https://opensearch-project.github.io/opensearch-k8s-operator/
version: 2.7.0
- name: opensearch-cluster
repository: https://opensearch-project.github.io/opensearch-k8s-operator/
version: 3.0.0
digest: sha256:90fbf8efd069b35ea52d4fd9babec479debcca1c199383a7f587f8720756743d
generated: "2024-11-27T15:48:06.046957+01:00"
23 changes: 23 additions & 0 deletions opensearch/chart/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# SPDX-FileCopyrightText: 2024 SAP SE or an SAP affiliate company and Greenhouse contributors
# SPDX-License-Identifier: Apache-2.0

apiVersion: v2
name: opensearch-operator
version: 2.6.0
description: A Helm chart for the OpenSearch operator
type: application
maintainers:
- name: timojohlo
- name: kuckkuck
- name: joluc
home: https://github.com/sapcc/helm-charts/tree/master/system/opensearch-operator
sources:
- https://github.com/opensearch-project/opensearch-k8s-operator
appVersion: 2.6.0
dependencies:
- name: opensearch-operator
repository: https://opensearch-project.github.io/opensearch-k8s-operator/ # TODO: Move to -> oci://ghcr.io/cloudoperators/greenhouse-extensions/charts
version: 2.7.0
- name: opensearch-cluster
repository: https://opensearch-project.github.io/opensearch-k8s-operator/ # TODO: Move to -> oci://ghcr.io/cloudoperators/greenhouse-extensions/charts
version: 3.0.0
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# SPDX-FileCopyrightText: 2024 SAP SE or an SAP affiliate company and Greenhouse contributors
# SPDX-License-Identifier: Apache-2.0

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.14.0
name: opensearchactiongroups.opensearch.opster.io
spec:
group: opensearch.opster.io
names:
kind: OpensearchActionGroup
listKind: OpensearchActionGroupList
plural: opensearchactiongroups
shortNames:
- opensearchactiongroup
singular: opensearchactiongroup
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
description: OpensearchActionGroup is the Schema for the opensearchactiongroups
API
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: OpensearchActionGroupSpec defines the desired state of OpensearchActionGroup
properties:
allowedActions:
items:
type: string
type: array
description:
type: string
opensearchCluster:
description: |-
LocalObjectReference contains enough information to let you locate the
referenced object inside the same namespace.
properties:
name:
description: |-
Name of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?
type: string
type: object
x-kubernetes-map-type: atomic
type:
type: string
required:
- allowedActions
- opensearchCluster
type: object
status:
description: OpensearchActionGroupStatus defines the observed state of
OpensearchActionGroup
properties:
existingActionGroup:
type: boolean
managedCluster:
description: |-
UID is a type that holds unique ID values, including UUIDs. Because we
don't ONLY use UUIDs, this is an alias to string. Being a type captures
intent and helps make sure that UIDs and names do not get conflated.
type: string
reason:
type: string
state:
type: string
type: object
type: object
served: true
storage: true
subresources:
status: {}
Loading
Loading