Skip to content

Commit

Permalink
Add ResourceLimits for test pods
Browse files Browse the repository at this point in the history 
The test pods spawned by the test operator prior to this change were
executed without any resource limits. This ultimately meant that the
pods could consume an unlimited amount of resources until the limit for
a worker node was reached.

This commit introduces default resource limits for each test pod
spawned by the test operator (tempest, tobiko, ansibletest, horizontest
pod). The default value can be overridden using .Spec.Resources
parameter which is introduced as part of this commit as well [1]:

spec:
  resources:
    requests:
      memory: 1Gi
      cpu:    250m
    limits:
      memory: 2Gi
      cpu:    500M

[1] https://pkg.go.dev/k8s.io/api/core/v1#ResourceRequirements

Depends-On: openstack-k8s-operators/ci-framework#2617
  • Loading branch information
@lpiwowar
lpiwowar committed Dec 18, 2024
1 parent 20927e1 commit 8e66c54
Show file tree
Hide file tree
Showing 22 changed files with 864 additions and 7 deletions.
114 changes: 114 additions & 0 deletions api/bases/test.openstack.org_ansibletests.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,62 @@ spec:
to work properly (e.g.: extraRPMs in Tempest CR, or certain set
of tobiko tests).'
type: boolean
resources:
default:
limits:
cpu: 4000m
memory: 4Gi
requests:
cpu: 2000m
memory: 2Gi
description: The desired amount of resources that should be assigned
to each test pod spawned using the AnsibleTest CR. https://pkg.go.dev/k8s.io/api/core/v1#ResourceRequirements
properties:
claims:
description: "Claims lists the names of resources, defined in
spec.resourceClaims, that are used by this container. \n This
is an alpha field and requires enabling the DynamicResourceAllocation
feature gate. \n This field is immutable. It can only be set
for containers."
items:
description: ResourceClaim references one entry in PodSpec.ResourceClaims.
properties:
name:
description: Name must match the name of one entry in pod.spec.resourceClaims
of the Pod where this field is used. It makes that resource
available inside a container.
type: string
required:
- name
type: object
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources
allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. Requests cannot exceed Limits.
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
storageClass:
default: local-storage
description: StorageClass used to create any test-operator related
Expand Down Expand Up @@ -290,6 +346,64 @@ spec:
test-operator functionalities to work properly (e.g.: extraRPMs
in Tempest CR, or certain set of tobiko tests).'
type: boolean
resources:
default:
limits:
cpu: 2000m
memory: 2Gi
requests:
cpu: 1000m
memory: 2Gi
description: The desired amount of resources that should be
assigned to each test pod spawned using the AnsibleTest CR.
https://pkg.go.dev/k8s.io/api/core/v1#ResourceRequirements
properties:
claims:
description: "Claims lists the names of resources, defined
in spec.resourceClaims, that are used by this container.
\n This is an alpha field and requires enabling the DynamicResourceAllocation
feature gate. \n This field is immutable. It can only
be set for containers."
items:
description: ResourceClaim references one entry in PodSpec.ResourceClaims.
properties:
name:
description: Name must match the name of one entry
in pod.spec.resourceClaims of the Pod where this
field is used. It makes that resource available
inside a container.
type: string
required:
- name
type: object
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified,
otherwise to an implementation-defined value. Requests
cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
stepName:
description: Name of a workflow step. The step name will be
used for example to create a logs directory.
Expand Down
56 changes: 56 additions & 0 deletions api/bases/test.openstack.org_horizontests.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,62 @@ spec:
default: https://review.opendev.org/openstack/horizon
description: RepoUrl is the URL of the Horizon repository.
type: string
resources:
default:
limits:
cpu: 2000m
memory: 4Gi
requests:
cpu: 1000m
memory: 2Gi
description: The desired amount of resources that should be assigned
to each test pod spawned using the HorizonTest CR. https://pkg.go.dev/k8s.io/api/core/v1#ResourceRequirements
properties:
claims:
description: "Claims lists the names of resources, defined in
spec.resourceClaims, that are used by this container. \n This
is an alpha field and requires enabling the DynamicResourceAllocation
feature gate. \n This field is immutable. It can only be set
for containers."
items:
description: ResourceClaim references one entry in PodSpec.ResourceClaims.
properties:
name:
description: Name must match the name of one entry in pod.spec.resourceClaims
of the Pod where this field is used. It makes that resource
available inside a container.
type: string
required:
- name
type: object
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources
allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. Requests cannot exceed Limits.
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
storageClass:
default: local-storage
description: StorageClass used to create any test-operator related
Expand Down
113 changes: 113 additions & 0 deletions api/bases/test.openstack.org_tempests.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,62 @@ spec:
to work properly (e.g.: extraRPMs in Tempest CR, or certain set
of tobiko tests).'
type: boolean
resources:
default:
limits:
cpu: 8000m
memory: 4Gi
requests:
cpu: 4000m
memory: 2Gi
description: The desired amount of resources that should be assigned
to each test pod spawned using the Tempest CR. https://pkg.go.dev/k8s.io/api/core/v1#ResourceRequirements
properties:
claims:
description: "Claims lists the names of resources, defined in
spec.resourceClaims, that are used by this container. \n This
is an alpha field and requires enabling the DynamicResourceAllocation
feature gate. \n This field is immutable. It can only be set
for containers."
items:
description: ResourceClaim references one entry in PodSpec.ResourceClaims.
properties:
name:
description: Name must match the name of one entry in pod.spec.resourceClaims
of the Pod where this field is used. It makes that resource
available inside a container.
type: string
required:
- name
type: object
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute resources
allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. Requests cannot exceed Limits.
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
storageClass:
default: local-storage
description: StorageClass used to create any test-operator related
Expand Down Expand Up @@ -584,6 +640,63 @@ spec:
test-operator functionalities to work properly (e.g.: extraRPMs
in Tempest CR, or certain set of tobiko tests).'
type: boolean
resources:
default:
limits:
cpu: 8000m
memory: 4Gi
requests:
cpu: 4000m
memory: 2Gi
description: The desired amount of resources that should be
assigned to each test pod spawned using the Tempest CR. https://pkg.go.dev/k8s.io/api/core/v1#ResourceRequirements
properties:
claims:
description: "Claims lists the names of resources, defined
in spec.resourceClaims, that are used by this container.
\n This is an alpha field and requires enabling the DynamicResourceAllocation
feature gate. \n This field is immutable. It can only
be set for containers."
items:
description: ResourceClaim references one entry in PodSpec.ResourceClaims.
properties:
name:
description: Name must match the name of one entry
in pod.spec.resourceClaims of the Pod where this
field is used. It makes that resource available
inside a container.
type: string
required:
- name
type: object
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Limits describes the maximum amount of compute
resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: 'Requests describes the minimum amount of compute
resources required. If Requests is omitted for a container,
it defaults to Limits if that is explicitly specified,
otherwise to an implementation-defined value. Requests
cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
type: object
stepName:
description: Name of a workflow step. The step name will be
used for example to create a logs directory.
Expand Down
Loading

0 comments on commit 8e66c54

Please sign in to comment.