Skip to content

Commit 10cdfb3

Browse files
fixup! Update KEP 4381 with standard device attribute definitions.
1 parent 842945b commit 10cdfb3

File tree

1 file changed

+24
-15
lines changed
  • keps/sig-node/4381-dra-structured-parameters

1 file changed

+24
-15
lines changed

keps/sig-node/4381-dra-structured-parameters/README.md

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -502,12 +502,15 @@ In production, a similar PodTemplateSpec in a Deployment will be used.
502502
#### Co-locating devices based on hardware topology
503503

504504
As a user, I want to easily request multiple devices (like a CPU and a NIC) that
505-
are physically related in the system's hardware topology,
505+
are physically connected to the same CPU in the system's hardware.
506+
507+
For instance, in a multi-CPU (also known as multi-socket) system, I'd prefer
508+
that a requested NIC is attached to the CPU being allocated to improve
509+
performance by avoiding costly inter-CPU interconnects.
506510

507511
I'll define a `ResourceClaim` for my workload, specifying constraints with
508512
`MatchAttribute` to ensure the devices share the same underlying hardware
509-
characteristics. For instance, if I need a CPU and a NIC that minimize
510-
communication latency, I'd want them on the same NUMA node:
513+
characteristics
511514

512515
```
513516
apiVersion: resource.k8s.io/v1beta1
@@ -523,13 +526,13 @@ spec:
523526
deviceClassName: nic.vendor2.com # This is a hypothetical NIC device class
524527
constraints:
525528
- requests: ["cpu-request", "nic-request"]
526-
matchAttribute: k8s.io/numaNode
529+
matchAttribute: k8s.io/cpuSocketNumber
527530
```
528531
529532
I'll use one of the standard attributes provided by Kubernetes, choosing between
530-
`k8s.io/numaNode` or `k8s.io/pcieRoot` depending on my specific alignment needs.
531-
I know that even if my CPU and NIC are managed by different DRA drivers, they
532-
are likely publishing the information I can use for alignment through these
533+
`k8s.io/cpuSocketNumber` or `k8s.io/pcieRoot` depending on my specific alignment
534+
needs. I know that even if my CPU and NIC are managed by different DRA drivers,
535+
they are likely publishing the information I can use for alignment through these
533536
standard attributes.
534537
535538
### Publishing node resources
@@ -616,9 +619,10 @@ We are reserving the `k8s.io/` domain (and subdomains) prefix for attributes and
616619
capacities for standardization by the Kubernetes project. This reservation
617620
allows us to define common attributes that can describe hardware characteristics
618621
across resources from different vendors. Currently, we are defining two such
619-
standard attributes: `k8s.sio/numaNode` and `k8s.io/pcieRoot`. Details on their
620-
meaning and how they should be exposed by DRA drivers are available in the [API
621-
design section under ResourceSlice's](#resourceslice) QualifiedName definition.
622+
standard attributes: `k8s.io/cpuSocketNumber` and `k8s.io/pcieRoot`. Details on
623+
their meaning and how they should be exposed by DRA drivers are available in the
624+
[API design section under ResourceSlice's](#resourceslice) QualifiedName
625+
definition.
622626

623627
**Note:** If a driver needs to remove a device or change its attributes,
624628
then there is a risk that a claim gets allocated based on the old
@@ -1270,11 +1274,16 @@ const ResourceSliceMaxAttributesAndCapacitiesPerDevice = 32
12701274
//
12711275
// Currently, the two standard attributes are:
12721276
//
1273-
// 1. `k8s.io/numaNode`: An integer value referring to a Non-Uniform Memory
1274-
// Access (NUMA) node within the system's NUMA topology. This attribute can
1275-
// be used to describe which NUMA node a device is physically associated
1276-
// with. DRA drivers MAY discover this value for PCI devices via
1277-
// sysfs, for example, by reading `/sys/bus/pci/devices/<PCI_ADDRESS>/numa_node`.
1277+
// 1. `k8s.io/cpuSocketNumber`: An integer value referring to the logical
1278+
// identifier assigned by the operating system for the physical CPU socket
1279+
// that a device is associated with. For a PCIe device, DRA drivers can
1280+
// determine this value by first reading its associated NUMA node from
1281+
// `/sys/bus/pci/devices/<PCI_ADDRESS>/numa_node`. Then, using that NUMA
1282+
// node, the value for `cpuSocketNumber` can be found by reading the
1283+
// `physical_package_id` from any CPU within that node (e.g.,
1284+
// `/sys/devices/system/node<NUMA_NODE>/cpuX/topology/physical_package_id`).
1285+
// Similarly, for a logical CPU X, its `cpuSocketNumber` can be identified
1286+
// from `/sys/devices/system/cpu/cpuX/topology/physical_package_id`.
12781287
//
12791288
// 2. `k8s.io/pcieRoot`: A string value in the format `pci<domain>:<bus>`,
12801289
// referring to a PCIe (Peripheral Component Interconnect Express) Root

0 commit comments

Comments
 (0)