Gdb 6521 as a dev ops i want to be able to deploy graph db 10.0 using…

… helm (#37)

* Changes to graphdb-node.yaml from master to node

* Fixed up persistence

* Removed .graphdb.topology from values, 

* Exposed RPC server port.

* Implementing automatic cluster creation

* Fixed cluster creation

* Moved to real statefulset

* Added cluster config to values.yaml

* Fixed a bug with cluster-config.ttl

* Fixed usage of headless service

* Removed jolokia

* Graphdb-node-configmap is now provisioned.

* Changed node hostname

* Fixed cluster-config.json

* Fixed graphdb.sh to wait all nodes to be up

* Added delete cluster

* Removed GDB tools, 

* Changed management policy to parallel

* Added CPU limit

* Remove kong 

* Parse nodes count to int because of buggy helm cli causing issues during conversion

* Moved users.js and settings.js to data dir

* Changed the script for it to fail the post start job in cases where create/delete fails

* Changed the default cluster config to more sane values

* Add external cluster proxy to GraphDB 10 helm chart (#34)

* Cleaned up unnecessary hackery and services, external proxy configmap provisioning, removed unnecessary options

* Small fixes for external proxy configmap

* Fix to the external proxy yaml, increase proxy memory limit, clean up

* Remove service account

* Fixed external proxy not being able to connect to the graphdb instances

* Fixed external proxy not being able to communicate with GraphDB through rpc

* Health check for external proxy

* Edited script to not return error when there is already a cluster

* Changed default ingress timeouts

* Removed default repository

* values.yaml clean up

* Changed all the check with nodeCount to greater than rather not equal for cases when someone uses 0 nodes.

* Updated logback.xml

* Updated graphdb.properties, settings.js.

* Update README.md

* Cluster proxy moved to statefulset fixing issue when multiple are deployed

* Renamed the graphdb-cluster-proxy to gdb-proxy because of issues with longer namespace names

* Increased the post start job backoff limit to 9

* Added a value - graphdb.clusterConfig.clusterCreationTimeout to control the timeout of the create cluster curl query

* Added configurable liveness, readiness, startup probes

* Added more labels

* Removed default license. It will be provisioned when provided. Changed license location from conf to work directory.

* Added warning for lack of license when trying to make a cluster without providing one.

* Added probes to the README

* Added JDBC to README

* Updated migration guide

* Added configurable nodeSelector, affinity, tolerations, topologySpreadConstraints to graphdb-node and graphdb-external-proxy

* Removed the static persistance because it won't ever work with more than 1 replica

* Changed some services to headless so GKE can be happy

* Set GraphDB home for the graphdb-node, instead of relying on the Docker image

* Updated Changelog

Co-authored-by: Luben Karaslavov <[email protected]>
Co-authored-by: Nikolay Kolev <[email protected]>

CHANGELOG.md

@@ -1,4 +1,17 @@
 # GraphDB Helm chart release notes
+## Version 10.0
+
+### Breaking
+New major release that isn't compatible with the old chart, due to major breaking changes in Graphdb 10.
+Migration steps can be found [here](README.md#cluster-migration-from-graphdb-9x-to-100).
+
+### New
+- Changed to work with the new GraphDB 10.
+- Removed Kong.
+- Moved from multiple stateful sets with 1 replica to statefulsets with multiple replicas.
+- Configurable liveness, readiness, startup probes.
+- Can use standalone without license by default. Don't forget to set your license for a working cluster and connectors!
+- New overridable configmaps for users, settings and logback.
 
 ## Version 9.9.0
 

Chart.yaml

@@ -1,9 +1,9 @@
 #
-# Helm chart for GraphDB Free/SE/EE
+# Helm chart for GraphDB
 #
 apiVersion: v2
 name: graphdb
-description: Helm chart for GraphDB Free/SE/EE
+description: Helm chart for GraphDB
 type: application
 version: 10.0.0
 appVersion: 10.0.0

files/config/cluster-config.json

@@ -0,0 +1,13 @@
+{
+  "electionMinTimeout": {{ $.Values.graphdb.clusterConfig.electionMinTimeout }},
+  "electionRangeTimeout": {{ $.Values.graphdb.clusterConfig.electionRangeTimeout }},
+  "heartbeatInterval": {{ $.Values.graphdb.clusterConfig.heartbeatInterval }},
+  "messageSize": {{ $.Values.graphdb.clusterConfig.messageSize }},
+  "verificationTimeout": {{ $.Values.graphdb.clusterConfig.verificationTimeout }},
+  "nodes": [
+    {{- range $i, $node_index := until ( (int $.Values.graphdb.clusterConfig.nodesCount) )}}
+    "graphdb-node-{{ $node_index }}.graphdb-node.{{ $.Release.Namespace }}.svc.cluster.local:7300"{{- if gt (sub (int $.Values.graphdb.clusterConfig.nodesCount) 1 ) $node_index }},
+    {{- end }}
+    {{- end }}
+  ]
+}

files/config/graphdb.properties

@@ -42,6 +42,12 @@
 # graphdb.home.work =
 #
 #
+# GraphDB external plugins directory. This can be used to add
+# additional external plugins outside the distribution's lib/plugins directory.
+#
+# graphdb.extra.plugins =
+#
+#
 # Graphdb global page cache memory parameter. By default the page cache will
 # allocate 50% of the maximum Java heap memory (-Xmx) for the JVM process.
 # Setting this value too high will cause OME during the execution of memory intensive queries.
@@ -146,6 +152,12 @@
 # OpenID issuer URL, used to derive keys, endpoints and token validation. No default value.
 # graphdb.auth.openid.issuer = https://accounts.example.com
 #
+# OpenID well-known config URL, used to fetch the OpenID configuration of the OpenID provider.
+# The default value will be derived by appending /.well-known/openid-configuration to the issuer URL.
+# Use this setting if your OpenID does not conform to the specification and the well-known config is
+# served on a different URL than the default standard-mandated value.
+# graphdb.auth.openid.well_known_config_url = https://openid.example.com/custom/.well-known/openid-configuration
+#
 # OpenID client ID, used to authenticate and validate tokens. No default value.
 # graphdb.auth.openid.client_id = my-client-id
 #
@@ -166,12 +178,22 @@
 # OpenID expected audience in tokens, used to validate tokens. The default value is the same as the client ID.
 # graphdb.auth.openid.token_audience = my-audience
 #
+# OpenID extra scopes to request. Multiple scopes can be specified by separating them with a space.
+# By default GraphDB requests only the 'openid' scope and, if supported, the 'offline_access' scope.
+# Scopes are used to request sets of claims, e.g. you might need to set this to a provider-specific value
+# in order to obtain the username_name or the roles_claim (if using OAuth as well). The default value is empty.
+# graphdb.auth.openid.extra_scopes = profile email
+#
 # OpenID extra parameters for the authorize endpoint. Some OpenID providers require additional parameters sent
 # to the authorize endpoint (e.g. resource=xxx). This is a URL encoded string where each parameter-value pair
 # is delimited by &. The string will be appended to the rest of the authorize URL parameters.
 # The default value is the empty string.
 # graphdb.auth.openid.authorize_parameters = param1=value%201&param2=value%202
 #
+# OpenID Oracle identity domain. Oracle Access Manager has a non-standard OpenID implementation that requires
+# an additional parameter: the Oracle identity domain name. If you use OAM set this to your identity domain name.
+# graphdb.auth.openid.oracle_domain = my-oracle-domain
+#
 # OpenID use GraphDB as proxy for the JWKS URL and token endpoints. This can be used to bypass an OpenID provider
 # without a proper CORS configuration. The value is a boolean true/false. False by default.
 # graphdb.auth.openid.proxy = false
@@ -187,6 +209,10 @@
 # Enable OAuth authorization. The default value is 'local' corresponding to local users.
 # graphdb.auth.database = oauth
 #
+# Set case-insensitive validation for user accounts so that users can log in regardless of the case used at login time.
+# The default value is false.
+# graphdb.auth.database.case_insensitive = false
+#
 # OpenID issuer URL, used to derive keys, endpoints and token validation. No default value.
 # graphdb.auth.openid.issuer = https://accounts.example.com
 #
@@ -227,6 +253,10 @@
 # The prefix will be stripped when the roles are mapped. The default value is the empty string.
 # graphdb.auth.oauth.roles_prefix = GDB_
 #
+# OAuth roles suffix to strip. The roles claim may provide the GraphDB roles with some suffix, e.g. ROLE_USER_GDB.
+# The suffix will be stripped when the roles are mapped. The default value is the empty string.
+# graphdb.auth.oauth.roles_suffix = ROLE_USER_GDB
+#
 # OAuth default roles to assign. It may be convenient to always assign certain roles without listing them in the roles
 # claim. The value is a comma-delimited list of GraphDB roles. The default value is the empty list.
 #
@@ -241,7 +271,7 @@
 # graphdb.auth.database = ldap
 # graphdb.auth.ldap.url = ldap://localhost:10389/dc=example,dc=org
 
-# Permit access for all users that are part of the “people” unit of the fictional “example.org” organisation.
+# Permit access for all users that are part of the \u201Cpeople\u201D unit of the fictional \u201Cexample.org\u201D organisation.
 #
 # graphdb.auth.ldap.user.search.base = ou=people
 # graphdb.auth.ldap.user.search.filter = (cn={0})
@@ -329,3 +359,15 @@
 # the request it will be generated randomly in UUID type 5 format.
 #
 # graphdb.append.request.id.headers = true
+
+###### HEAP DUMPS ######
+#
+# GraphDB can dump the heap on out of memory errors in order to provide insight to the cause
+# for excessive memory usage.
+#
+# Enable or disable the heap dump (enabled by default)
+# graphdb.heapdump.enable = true
+#
+# File to write the heap dump to, by default this is the file heapdump.hprof in the configured logs directory.
+# See also the properties graphdb.home and graphdb.home.logs.
+# graphdb.heapdump.path =