Skip to content

Commit c96efd1

Browse files
committed
remove deviceIndex
Signed-off-by: Alay Patel <[email protected]>
1 parent 6324114 commit c96efd1

File tree

1 file changed

+20
-43
lines changed
  • keps/sig-node/5304-dra-attributes-downward-api

1 file changed

+20
-43
lines changed

keps/sig-node/5304-dra-attributes-downward-api/README.md

Lines changed: 20 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -96,12 +96,20 @@ tags, and then generate with `hack/update-toc.sh`.
9696
- [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional)
9797
- [Risks and Mitigations](#risks-and-mitigations)
9898
- [Design Details](#design-details)
99+
- [API Changes](#api-changes)
100+
- [New Downward API Selector: ResourceSliceAttributeSelector](#new-downward-api-selector-resourcesliceattributeselector)
101+
- [Kubelet Implementation](#kubelet-implementation)
102+
- [Usage Examples](#usage-examples)
103+
- [Feature Gate](#feature-gate)
99104
- [Test Plan](#test-plan)
100105
- [Prerequisite testing updates](#prerequisite-testing-updates)
101106
- [Unit tests](#unit-tests)
102107
- [Integration tests](#integration-tests)
103108
- [e2e tests](#e2e-tests)
104109
- [Graduation Criteria](#graduation-criteria)
110+
- [Alpha (v1.35)](#alpha-v135)
111+
- [Beta](#beta)
112+
- [GA](#ga)
105113
- [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
106114
- [Version Skew Strategy](#version-skew-strategy)
107115
- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
@@ -193,7 +201,7 @@ This proposal introduces a new Downward API selector (`resourceSliceAttributeRef
193201
4. Maintains a per-Pod cache of `(claimName, requestName) -> {attribute: value}` mappings
194202
5. Resolves `resourceSliceAttributeRef` references when containers start
195203

196-
Downward API references expose one attribute per reference. The kubelet resolves only the attribute explicitly referenced via `resourceSliceAttributeRef`.
204+
Downward API references expose one attribute per reference. The kubelet resolves only the attribute explicitly referenced via `ResourceSliceAttributeSelector`.
197205

198206
### User Stories (Optional)
199207

@@ -251,14 +259,6 @@ type ResourceSliceAttributeSelector struct {
251259
// The attribute name must be present in the ResourceSlice's device attributes.
252260
// +required
253261
Attribute string `json:"attribute"`
254-
255-
// DeviceIndex selects which device's attribute to surface when a request
256-
// is satisfied by multiple devices. When unset, attributes from all
257-
// allocated devices for the request are joined in allocation order.
258-
// Zero-based index into the allocation results for the matching request.
259-
// Must be >= 0 if set. Bounds are validated at resolution time by the kubelet.
260-
// +optional
261-
DeviceIndex *int32 `json:"deviceIndex,omitempty"`
262262
}
263263

264264
// In core/v1 EnvVarSource:
@@ -278,7 +278,7 @@ Validation:
278278
- Enforce exactly one of `fieldRef`, `resourceFieldRef`, or `resourceSliceAttributeRef` in both env and volume items
279279
- Validate `claimName` and `requestName` against DNS label rules
280280
- No API-level enumeration of attribute names; kubelet resolves attributes that exist in the matching `ResourceSlice` at runtime
281-
- If `deviceIndex` is set, validate it is >= 0; if unset, aggregate across all allocated devices
281+
282282

283283
####
284284

@@ -292,14 +292,14 @@ The kubelet runs a local DRA attributes controller that:
292292
4. Maintains Cache: Keeps a per-Pod map of `(claimName, requestName) -> {attribute: value}` with a readiness flag
293293

294294
Resolution Semantics:
295+
- Prioritized List compatibility:
296+
- Clients do not need to know the number of devices a priori. At container start, kubelet aggregates the attribute across all devices actually allocated for the request and joins the values with "," in allocation order. If any allocated device lacks the attribute, resolution fails and the pod start errors.
295297
- Cache entries are updated on claim/slice changes
296298
- For container environment variables, resolution happens at container start using the latest ready values
297-
- Attributes are not frozen at allocation time; scheduler and controllers are not involved in copying attributes
298-
299-
- Failure on missing data: If the `ResourceSlice` is not found, the attribute is absent, or `deviceIndex` is out of range at container start, kubelet records a warning event and returns an error to the sync loop. The pod start fails per standard semantics (e.g., `restartPolicy` governs restarts; Jobs will fail the pod).
300-
- Multi-device requests:
301-
- When `deviceIndex` is unset, kubelet resolves the attribute across all allocated devices for the request, preserving allocation order, and joins values with a comma (",") into a single string. Devices that do not report the attribute are skipped. If no devices provide the attribute, the value is considered not ready.
302-
- When `deviceIndex` is set, kubelet selects the device at that zero-based index from the allocation results and resolves the attribute for that device only. If the index is out of range or the attribute is missing on that device, the value is considered not ready.
299+
- Attributes are not frozen at allocation time; scheduler and controllers are not involved in copying attributes
300+
301+
- Failure on missing data: If the `ResourceSlice` is not found, or the attribute is absent on any allocated device at container start, kubelet records a warning event and returns an error to the sync loop. The pod start fails per standard semantics (e.g., `restartPolicy` governs restarts; Jobs will fail the pod).
302+
- Multi-device requests: Kubelet resolves the attribute across all allocated devices for the request, preserving allocation order, and joins values with a comma (",") into a single string. If any allocated device does not report the attribute, resolution fails (pod start error).
303303

304304
Security & RBAC:
305305
- Node kubelet uses NodeAuthorizer to watch/read `ResourceClaim` and `ResourceSlice` objects related to Pods scheduled to the node
@@ -329,34 +329,12 @@ spec:
329329
claimName: pgpu-claim
330330
requestName: pgpu-request
331331
attribute: resource.kubernetes.io/pcieRoot
332-
# deviceIndex omitted -> aggregate across all devices
332+
# If multiple devices are allocated for this request, values are joined with "," in allocation order.
333333
```
334334

335335

336336

337-
Environment Variable Example (SpecificIndex for multi-device request):
338-
339-
```yaml
340-
apiVersion: v1
341-
kind: Pod
342-
metadata:
343-
name: virt-launcher-gpu-index
344-
spec:
345-
resourceClaims:
346-
- name: pgpu-claim
347-
resourceClaimName: my-physical-gpu-claim
348-
containers:
349-
- name: compute
350-
image: virt-launcher:latest
351-
env:
352-
- name: PGPU_CLAIM_PCI_ROOT_INDEX1
353-
valueFrom:
354-
resourceSliceAttributeRef:
355-
claimName: pgpu-claim
356-
requestName: pgpu-request
357-
attribute: resource.kubernetes.io/pcieRoot
358-
deviceIndex: 1
359-
```
337+
360338

361339
### Feature Gate
362340

@@ -406,8 +384,8 @@ Integration tests will cover:
406384

407385
- Feature gate toggling: Verify API rejects `resourceSliceAttributeRef` when feature gate is disabled
408386
- End-to-end resolution: Create Pod with resourceClaims, verify env vars contain correct attribute values
409-
- Negative cases: Missing allocation, missing `ResourceSlice`, missing attribute, invalid `deviceIndex` — expect warning event and pod start failure
410-
- Multi-device semantics: All-mode joining order and delimiter; SpecificIndex with valid/invalid index; missing attribute on selected device
387+
- Negative cases: Missing allocation, missing `ResourceSlice`, missing attribute on any allocated device — expect warning event and pod start failure
388+
- Multi-device semantics: Joining order and delimiter; mixed presence of attributes across allocated devices should cause failure
411389

412390
Tests will be added to `test/integration/kubelet/` and `test/integration/dra/`.
413391

@@ -430,7 +408,6 @@ Tests will be added to `test/e2e/dra/` and `test/e2e_node/downwardapi_test.go`.
430408
- Feature implemented behind `DRADownwardDeviceAttributes` feature gate
431409
- API types added: `resourceSliceAttributeRef` in `core/v1.EnvVarSource`
432410
- Kubelet DRA attributes controller implemented
433-
- Support for `resource.kubernetes.io/pcieRoot` and `dra.kubervirt.io/mdevUUID` attributes
434411
- Unit tests for validation, cache, and resolution logic
435412
- Initial integration and e2e tests completed
436413
- Documentation published for API usage

0 commit comments

Comments
 (0)