spark-operator-1.0.1

A Helm chart for Spark on Kubernetes operator

spark-operator-1.0.0

A Helm chart for Spark on Kubernetes operator

Release v1beta2-1.2.1 with Spark 3.0.0

This release includes the following notable enhancements and bug fixes:

• Added support for custom request resources in the Volcano scheduler integration: #966.
• Extended ingress URL format for the Spark driver UI: #1008.
• Added support for proxy users: #1022.
• Added support for container-level security context: #1023.
• Added support for spark-submit packages and repositories: #1040.

Release v1beta2-1.2.0 with Spark 3.0.0

This is the first release based on the new Spark 3.0.0 release. Changes in this release include support for some new configuration options and enhancements introduced in Spark 3.0.0 that are relevant to the operator. Major commits in this release include #954 and #976. The minor version is bumped to reflect that it is based on Spark 3.0.0.

Release v1beta2-1.1.2 with Spark 2.4.5

This is a bug fix release based on Spark 2.4.5. Notable fixes/enhancements include:

• #935: added the container port for exposing Spark metrics in Prometheus data format.
• #914: support customization of Spark driver UI port and ingress configuration.
• #898: fixed how default values of certain fields get applied.
• #876: fix for #868.
• #867: fix for termination time when using sidecars.
• #852: added a new metric on job start latency.
• #847: fix around use of metricsProperties and hasPrometheusConfigFile.
• Other fixes and refactoring.

Release v1beta2-1.1.1 with Spark 2.4.5

This is a bug fix release, which includes a fix in #832 that caused issues reported in #795 and #826.

Release v1beta2-1.1.0 with Spark 2.4.5

This release is based on Spark 2.4.5 and API version v1beta2, and includes a number of bug fixes and enhancements.

Release v1beta2-1.0.1 with Spark 2.4.4

This is a patch release and contains important bug fixes in #645, #647, and #649.

Release v1beta2-1.0.0 with Spark 2.4.4

This is the first release with the new v1beta2 version of the API (#578). This release also includes an upgrade to Spark 2.4.4 (#625) for both the operator image and Spark images for the various language bindings. This release includes the following enhancements and bug fixes:

• Support for high-availability mode with leader election (#518, #575).
• Allow PodSecurityContext to be overridden (#528).
• Spark pod mutating webhook enhancements (#543).
• A new ResourceQuota enforcement webhook (#544 ).
• Switch to using controller-gen to generate/update CRD yamls (#578).
• Integration with Volcano for batch scheduling (#567, #585, #599, #606).
• TTL support for SparkApplications (#615).
• Bug fixes.
• Documentation enhancements.

Release v1beta1-0.9.0

This release includes the following enhancements and fixes:

• Support for adding sidecar containers into the driver and executor pods (#478).
• Support for requesting GPUs (#509).
• Support for specifying hostNetwork for the driver and executor pods (#510).
• Support for specifying node selectors at pod level (#504).
• Support for specifying DNS config options for the driver and executor pods (#519).
• Changed the way driver pod status is checked (#506). The operator now sets the name of the driver pod while submitting an application to run and uses the name to get the driver pod instead of listing the Spark pods and filter out the driver pod.
• Changed the type of the driver UI service from NodePort to ClusterIP (#520).
• Other fixes and documentation enhancements.