Release notes for 24.3

modules/ROOT/pages/release-notes.adoc

@@ -45,7 +45,7 @@ Here are the headings you can use for the next release. Saves time checking inde
 
 Support for many new product versions::
 Almost all product images have been updated to their latest versions.
-Some notable examples are Apache Airflow 2.8.1, Trino 422 and Apache Spark 3.5.1.
+Some notable examples are Apache Airflow 2.8.1, Trino 442 and Apache Spark 3.5.1.
 In addition, we started building some product binaries from source instead of packaging them from the official releases.
 This enables greater control over the features and the security aspects of the products.
 Currently, Apache Hadoop and Apache HBase are built from source and others will follow in coming releases.
@@ -62,7 +62,7 @@ Starting with this release, user **authorization** is consistent across all prod
 The Open Policy Agent (OPA) has been a core component of the platform since the beginning and in this release we enhanced its capability with the inclusion of a new component called `user-info-fetcher`.
 It allows for authorization policies to be built on many attributes like organizational group membership, resource assignment and much more.
 The first major identity provider supported by the `user-info-fetcher` is Keycloak but others will follow.
-TODO: Link to documentation
+The documentation is available xref:opa:usage-guide/user-info-fetcher.adoc[here]
 
 A lot of effort was spent on enabling policy based authorization (with OPA) within HDFS.
 This was in important milestone in the platform evolution and is a unique feature that has long been missed by HDFS users and administrators.
@@ -75,8 +75,7 @@ We added support for it in Apache Hive and Apache HBase.
 We also added examples for running Apache Spark applications in a Kerberos enabled environment.
 
 OpenID Connect is the de-facto authorization standard on the Web, and it's making its way into enterprise environments.
-Our platform now supports it for Apache Superset and Trino, others will follow.
-TODO: Link to docs
+Our platform now supports it for Apache Superset (xref:superset:usage-guide/security.adoc[documentation]) and Trino (xref:opa:usage-guide/security.adoc[documentation]), others will follow.
 
 A core component of SDP is the Secret Operator.
 In this release, the secret operator will automatically rotate certificates it generates.
@@ -98,7 +97,7 @@ HDFS deployments now support __rack awareness__.
 This is another unique feature that brings the SDP platform closer to feature parity with bare metal HDFS deployments.
 Of course, the exact meaning of __rack__ is different in Kubernetes environments, but the effect is the same: DataNodes are brought closer to the data they are reading and writing thus improving performance and reliability.
 A new https://github.com/stackabletech/hdfs-topology-provider[topology provider] is bundled with the HDFS image that maps Kubernetes labels to a cluster topology.
-TODO: Link to docs
+// TODO: Link to docs - apparently the README in in the linked repository is all there is.
 
 Documentation::
 We are constantly working on improving the platform documentation and custom resource definitions are a significant part of that.
@@ -144,9 +143,8 @@ The following are selected product features provided by new versions available i
   ** Apache Superset 3.1 includes various smaller new features/optimizations e.g. waterfall chart visualization, ECharts bubble chart, improved data set selectors, automatically format SQL queries, and country map visualization improvements.
 * Trino:
   ** Lots of improvements and optimization since release 428.
-  ** Most notably we would like to highlight support for access control with the Open Policy Agent that we ourselves contributed (s.a.) in release 438 (#19532).
+  ** Most notably we would like to highlight support for access control with the Open Policy Agent that we ourselves contributed in release 438 (#19532).
   ** Also, starting from release 440, there is now row filtering and column masking in Open Policy Agent.
-TODO: What is s.a.?
 * Apache ZooKeeper: Security and bug fixes.
 
 === Product versions
@@ -236,43 +234,10 @@ You will need to adapt your existing CRDs due to the following breaking changes
 .Breaking changes details
 [%collapsible]
 ====
-Apache Airflow requires a database to store metadata.
-The credentials for this database are configured using the `clusterConfig.credentialsSecret` property.
-This is `airflow-credentials` in the example below.
-
-[source,yaml]
-----
-apiVersion: airflow.stackable.tech/v1alpha1
-kind: AirflowCluster
-metadata:
-  name: airflow
-spec:
-  image:
-    productVersion: "2.8.1"
-  clusterConfig:
-    credentialsSecret: airflow-credentials
-----
-
-The Secret must contain credentials for Airflow `admin` user as well as connection properties to the database and Redis as shown below.
-
-[source,yaml]
-----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: airflow-credentials
-type: Opaque
-stringData:
-  adminUser.username: airflow
-  adminUser.firstname: Airflow
-  adminUser.lastname: Admin
-  adminUser.email: [email protected]
-  adminUser.password: airflow
-  connections.secretKey: thisISaSECRET_1234
-  connections.sqlalchemyDatabaseUri: postgresql+psycopg2://airflow:airflow@airflow-postgresql/airflow
-  connections.celeryResultBackend: db+postgresql://airflow:airflow@airflow-postgresql/airflow
-  connections.celeryBrokerUrl: redis://:redis@airflow-redis-master:6379/0
-----
+The following fields used to be optional but are now mandatory:
+* `spec.clusterConfig.credentialsSecret`: Name of the secret containing the credentials for the database.
+* `spec.clusterConfig.exposeConfig`: Set to `true` to export the `AIRFLOW__WEBSERVER__EXPOSE__CONFIG` environment variable.
+* `spec.clusterConfig.loadExamples`: Set to `true` to load example DAGs into the Airflow cluster.
 ====
 
 * https://github.com/stackabletech/airflow-operator/pull/366[Removed legacy node selector on roleGroups]
@@ -403,44 +368,8 @@ On the other hand, it enables dynamic provisioning of java packages (such as Del
 
 * https://github.com/stackabletech/superset-operator/pull/429[Fixed various issues in the CRD structure. `clusterConfig.credentialsSecret` is now mandatory]
 
-.Breaking changes details
-[%collapsible]
-====
-Apache Superset requires a database to store metadata.
-The credentials for this database must be provided with the `clusterConfig.credentialsSecret` property.
-
-For example, given the following Superset cluster snippet:
-[source,yaml]
-----
-apiVersion: superset.stackable.tech/v1alpha1
-kind: SupersetCluster
-metadata:
-  name: superset
-spec:
-  clusterConfig:
-    credentialsSecret: superset-credentials
-...
-----
-
-The Secret `superset-credentials` must contain the following fields:
-[source,yaml]
-----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: superset-credentials
-type: Opaque
-stringData:
-  adminUser.username: admin
-  adminUser.firstname: Superset
-  adminUser.lastname: Admin
-  adminUser.email: [email protected]
-  adminUser.password: admin
-  connections.secretKey: thisISaSECRET_1234
-  connections.sqlalchemyDatabaseUri: postgresql://superset:superset@superset-postgresql/superset
-----
-
-====
+The configuration for the Superset authentication, operations and listener class is specified within the `spec.clusterConfig` field.
+This field used to be optional but it is now required.
 
 
 === Upgrade from 23.11