Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - fix: Increase default pvc size to 1Gi as required by IONOS #365

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions docs/modules/hdfs/pages/usage-guide/resources.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,14 @@ The hdfs-operator will re-create the StatefulSet automatically.

include::home:concepts:stackable_resource_requests.adoc[]

If no resource requests are configured explicitly, the HDFS operator uses the following defaults:
A minimal HA setup consisting of 3 journalnodes, 2 namenodes and 2 datanodes has the following https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/[resource requirements]:

* `2950m` CPU request
* `8300m` CPU limit
* `6528m` memory request and limit
* `27648Mi` persistent storage

Corresponding to the values above, the operator uses the following resource defaults:

[source,yaml]
----
Expand All @@ -84,7 +91,7 @@ spec:
limit: 512Mi
storage:
data:
capacity: 512Mi
capacity: 1Gi
Techassi marked this conversation as resolved.
Show resolved Hide resolved
nameNodes:
config:
resources:
Expand All @@ -95,7 +102,7 @@ spec:
limit: 1024Mi
storage:
data:
capacity: 2048Mi
capacity: 2Gi
dataNodes:
config:
resources:
Expand Down
2 changes: 1 addition & 1 deletion rust/crd/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1138,7 +1138,7 @@ impl JournalNodeConfigFragment {
},
storage: HdfsStorageConfigFragment {
data: PvcConfigFragment {
capacity: Some(Quantity("512Mi".to_owned())),
capacity: Some(Quantity("1Gi".to_owned())),
Techassi marked this conversation as resolved.
Show resolved Hide resolved
storage_class: None,
selectors: None,
},
Expand Down
9 changes: 8 additions & 1 deletion rust/operator/src/container.rs
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,14 @@ impl ContainerConfig {
)])
// Only this init container needs the actual cert (from tls volume) to create the truststore + keystore from
.add_volume_mount("tls", "/stackable/tls")
.add_volume_mount("keystore", KEYSTORE_DIR_NAME);
.add_volume_mount("keystore", KEYSTORE_DIR_NAME)
.resources(ResourceRequirementsBuilder::new()
.with_cpu_request("100m")
.with_cpu_limit("400m")
.with_memory_request("512Mi")
.with_memory_limit("512Mi")
.build()
);

// We use the main app container resources here in contrast to several operators (which use
// hardcoded resources) due to the different code structure.
Expand Down
6 changes: 0 additions & 6 deletions tests/templates/kuttl/kerberos/10-install-zk.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,6 @@ spec:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
min: '300m'
max: '600m'
memory:
limit: '512Mi'
roleGroups:
default:
replicas: 1
18 changes: 0 additions & 18 deletions tests/templates/kuttl/kerberos/11-install-hdfs.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,6 @@ commands:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
max: '1'
min: '300m'
memory:
limit: '512Mi'
configOverrides: &configOverrides
core-site.xml:
hadoop.user.group.static.mapping.overrides: "dr.who=;nn=;nm=;jn=;testuser=supergroup;"
Expand All @@ -43,12 +37,6 @@ commands:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
max: '1'
min: '300m'
memory:
limit: '512Mi'
configOverrides: *configOverrides
roleGroups:
default:
Expand All @@ -57,12 +45,6 @@ commands:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
max: '1'
min: '300m'
memory:
limit: '512Mi'
configOverrides: *configOverrides
roleGroups:
default:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,6 @@ spec:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
min: '300m'
max: '600m'
memory:
limit: '512Mi'
roleGroups:
default:
replicas: 1
Expand Down
18 changes: 0 additions & 18 deletions tests/templates/kuttl/orphaned-resources/02-install-hdfs.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -17,25 +17,13 @@ spec:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
max: '1'
min: '300m'
memory:
limit: '512Mi'
roleGroups:
default:
replicas: 2
dataNodes:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
max: '1'
min: '300m'
memory:
limit: '512Mi'
roleGroups:
default:
replicas: 1
Expand All @@ -45,12 +33,6 @@ spec:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
max: '1'
min: '300m'
memory:
limit: '512Mi'
roleGroups:
default:
replicas: 1
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,6 @@ spec:
config:
logging:
enableVectorAgent: {{ lookup('env', 'VECTOR_AGGREGATOR') | length > 0 }}
resources:
cpu:
max: '1'
min: '300m'
memory:
limit: '512Mi'
roleGroups:
default: null
newrolegroup:
Expand Down