Skip to content

Commit

Permalink
Merge pull request #1409 from ConnorJC3/io2-iops
Browse files Browse the repository at this point in the history 
Add support for io2 Block Express volumes
  • Loading branch information
@k8s-ci-robot
k8s-ci-robot authored Oct 4, 2022
2 parents 74ab96e + d20afd3 commit eebda78
Show file tree
Hide file tree
Showing 5 changed files with 67 additions and 28 deletions.
34 changes: 18 additions & 16 deletions docs/parameters.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,27 @@ There are several optional parameters that may be passed into `CreateVolumeReque

The AWS EBS CSI Driver supports [tagging](tagging.md) through `StorageClass.parameters` (in v1.6.0 and later).

| Parameters | Values | Default | Description |
|-----------------------------|----------------------------------------------------------------|----------|---------------------|
| "csi.storage.k8s.io/fstype" | xfs, ext2, ext3, ext4 | ext4 | File system type that will be formatted during volume creation. This parameter is case sensitive! |
| "type" | io1, io2, gp2, gp3, sc1, st1, standard, sbp1, sbg1 | gp3* | EBS volume type. |
| "iopsPerGB" | | | I/O operations per second per GiB. Can be specified for IO1, IO2, and GP3 volumes. |
| "allowAutoIOPSPerGBIncrease"| true, false | false | When `"true"`, the CSI driver increases IOPS for a volume when `iopsPerGB * <volume size>` is too low to fit into IOPS range supported by AWS. This allows dynamic provisioning to always succeed, even when user specifies too small PVC capacity or `iopsPerGB` value. On the other hand, it may introduce additional costs, as such volumes have higher IOPS than requested in `iopsPerGB`.|
| "iops" | | | I/O operations per second. Can be specified for IO1, IO2, and GP3 volumes. |
| "throughput" | | 125 | Throughput in MiB/s. Only effective when gp3 volume type is specified. If empty, it will set to 125MiB/s as documented [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html). |
| "encrypted" | | | Whether the volume should be encrypted or not. Valid values are "true" or "false". |
| "kmsKeyId" | | | The full ARN of the key to use when encrypting the volume. If not specified, AWS will use the default KMS key for the region the volume is in. This will be an auto-generated key called `/aws/ebs` if not changed. |
| Parameters | Values | Default | Description |
|------------------------------|----------------------------------------------------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| "csi.storage.k8s.io/fstype" | xfs, ext2, ext3, ext4 | ext4 | File system type that will be formatted during volume creation. This parameter is case sensitive! |
| "type" | io1, io2, gp2, gp3, sc1, st1, standard, sbp1, sbg1 | gp3* | EBS volume type. |
| "iopsPerGB" | | | I/O operations per second per GiB. Can be specified for IO1, IO2, and GP3 volumes. |
| "allowAutoIOPSPerGBIncrease" | true, false | false | When `"true"`, the CSI driver increases IOPS for a volume when `iopsPerGB * <volume size>` is too low to fit into IOPS range supported by AWS. This allows dynamic provisioning to always succeed, even when user specifies too small PVC capacity or `iopsPerGB` value. On the other hand, it may introduce additional costs, as such volumes have higher IOPS than requested in `iopsPerGB`. |
| "iops" | | | I/O operations per second. Can be specified for IO1, IO2, and GP3 volumes. |
| "throughput" | | 125 | Throughput in MiB/s. Only effective when gp3 volume type is specified. If empty, it will set to 125MiB/s as documented [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html). |
| "encrypted" | true, false | false | Whether the volume should be encrypted or not. Valid values are "true" or "false". |
| "blockExpress" | true, false | false | Enables the creation of [io2 Block Express volumes](https://aws.amazon.com/ebs/provisioned-iops/#Introducing_io2_Block_Express) by increasing the IOPS limit for io2 volumes to 256000. Volumes created with more than 64000 IOPS will fail to mount on instances that do not support io2 Block Express. |
| "kmsKeyId" | | | The full ARN of the key to use when encrypting the volume. If not specified, AWS will use the default KMS key for the region the volume is in. This will be an auto-generated key called `/aws/ebs` if not changed. |

**Appendix**
* `gp3` is currently not supported on outposts. Outpost customers need to use a different type for their volumes.
* Unless explicitly noted, all parameters are case insensitive (e.g. "kmsKeyId", "kmskeyid" and any other combination of upper/lowercase characters can be used).
* If the requested IOPs (either directly from `iops` or from `iopsPerGB` multiplied by the volume's capacity) produces a value above the maximum IOPs allowed for the [volume type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html), the IOPS will be capped at the maximum value allowed. If the value is lower than the minimal supported IOPS value per volume, either an error is returned (the default behavior), or the value is increased to fit into the supported range when `allowautoiopspergbincrease` is `"true"`.
* If the requested IOPS (either directly from `iops` or from `iopsPerGB` multiplied by the volume's capacity) produces a value above the maximum IOPS allowed for the [volume type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html), the IOPS will be capped at the maximum value allowed. If the value is lower than the minimal supported IOPS value per volume, either an error is returned (the default behavior), or the value is increased to fit into the supported range when `allowautoiopspergbincrease` is `"true"`.
* You may specify either the "iops" or "iopsPerGb" parameters, not both. Specifying both parameters will result in an invalid StorageClass.

| Volume Type | Min total IOPS | Max total IOPS | Max IOPS per GB |
|-----------------------------|----------------------------------------|------------------|-------------------|
| IO1 | 100 | 64000 | 50 |
| IO2 | 100 | 64000 | 500 |
| GP3 | 3000 | 16000 | 500 |
| Volume Type | Min total IOPS | Max total IOPS | Max IOPS per GB |
|----------------------------|----------------|---------------|-------------------|
| io1 | 100 | 64000 | 50 |
| io2 (blockExpress = false) | 100 | 64000 | 500 |
| io2 (blockExpress = true) | 100 | 256000 | 500 |
| gp3 | 3000 | 16000 | 500 |
26 changes: 16 additions & 10 deletions pkg/cloud/cloud.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,15 +63,16 @@ const (
// AWS provisioning limits.
// Source: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html
const (
io1MinTotalIOPS = 100
io1MaxTotalIOPS = 64000
io1MaxIOPSPerGB = 50
io2MinTotalIOPS = 100
io2MaxTotalIOPS = 64000
io2MaxIOPSPerGB = 500
gp3MaxTotalIOPS = 16000
gp3MinTotalIOPS = 3000
gp3MaxIOPSPerGB = 500
io1MinTotalIOPS = 100
io1MaxTotalIOPS = 64000
io1MaxIOPSPerGB = 50
io2MinTotalIOPS = 100
io2MaxTotalIOPS = 64000
io2BlockExpressMaxTotalIOPS = 256000
io2MaxIOPSPerGB = 500
gp3MaxTotalIOPS = 16000
gp3MinTotalIOPS = 3000
gp3MaxIOPSPerGB = 500
)

var (
Expand Down Expand Up @@ -192,6 +193,7 @@ type DiskOptions struct {
AvailabilityZone string
OutpostArn string
Encrypted bool
BlockExpress bool
// KmsKeyID represents a fully qualified resource name to the key to use for encryption.
// example: arn:aws:kms:us-east-1:012345678910:key/abcd1234-a123-456a-a12b-a123b4cd56ef
KmsKeyID string
Expand Down Expand Up @@ -316,7 +318,11 @@ func (c *cloud) CreateDisk(ctx context.Context, volumeName string, diskOptions *
minIops = io1MinTotalIOPS
maxIopsPerGb = io1MaxIOPSPerGB
case VolumeTypeIO2:
maxIops = io2MaxTotalIOPS
if diskOptions.BlockExpress {
maxIops = io2BlockExpressMaxTotalIOPS
} else {
maxIops = io2MaxTotalIOPS
}
minIops = io2MinTotalIOPS
maxIopsPerGb = io2MaxIOPSPerGB
case VolumeTypeGP3:
Expand Down
20 changes: 20 additions & 0 deletions pkg/cloud/cloud_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,26 @@ func TestCreateDisk(t *testing.T) {
},
expErr: nil,
},
{
name: "success: large io2 Block Express with too high iopsPerGB",
volumeName: "vol-test-name",
diskOptions: &DiskOptions{
CapacityBytes: util.GiBToBytes(3333),
Tags: map[string]string{VolumeNameTagKey: "vol-test", AwsEbsDriverTagKey: "true"},
VolumeType: VolumeTypeIO2,
IOPSPerGB: 100000,
BlockExpress: true,
},
expDisk: &Disk{
VolumeID: "vol-test",
CapacityGiB: 3333,
AvailabilityZone: defaultZone,
},
expCreateVolumeInput: &ec2.CreateVolumeInput{
Iops: aws.Int64(256000),
},
expErr: nil,
},
{
name: "success: create volume when zone is snow and add tags",
volumeName: "vol-test-name",
Expand Down
3 changes: 3 additions & 0 deletions pkg/driver/constants.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,9 @@ const (
// provisioned volume
PVNameKey = "csi.storage.k8s.io/pv/name"

// BlockExpressKey increases the iops limit for io2 volumes to the block express limit
BlockExpressKey = "blockExpress"

// TagKeyPrefix contains the prefix of a volume parameter that designates it as
// a tag to be attached to the resource
TagKeyPrefix = "tagSpecification"
Expand Down
12 changes: 10 additions & 2 deletions pkg/driver/controller.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ func (d *controllerService) CreateVolume(ctx context.Context, req *csi.CreateVol
iops int
throughput int
isEncrypted bool
blockExpress bool
kmsKeyID string
scTags []string
volumeTags = map[string]string{
Expand Down Expand Up @@ -160,8 +161,6 @@ func (d *controllerService) CreateVolume(ctx context.Context, req *csi.CreateVol
case EncryptedKey:
if value == "true" {
isEncrypted = true
} else {
isEncrypted = false
}
case KmsKeyIDKey:
kmsKeyID = value
Expand All @@ -174,6 +173,10 @@ func (d *controllerService) CreateVolume(ctx context.Context, req *csi.CreateVol
case PVNameKey:
volumeTags[PVNameTag] = value
tProps.PVName = value
case BlockExpressKey:
if value == "true" {
blockExpress = true
}
default:
if strings.HasPrefix(key, TagKeyPrefix) {
scTags = append(scTags, value)
Expand All @@ -189,6 +192,10 @@ func (d *controllerService) CreateVolume(ctx context.Context, req *csi.CreateVol
}
}

if blockExpress && volumeType != cloud.VolumeTypeIO2 {
return nil, status.Errorf(codes.InvalidArgument, "Block Express is only supported on io2 volumes")
}

snapshotID := ""
volumeSource := req.GetVolumeContentSource()
if volumeSource != nil {
Expand Down Expand Up @@ -241,6 +248,7 @@ func (d *controllerService) CreateVolume(ctx context.Context, req *csi.CreateVol
AvailabilityZone: zone,
OutpostArn: outpostArn,
Encrypted: isEncrypted,
BlockExpress: blockExpress,
KmsKeyID: kmsKeyID,
SnapshotID: snapshotID,
}
Expand Down

0 comments on commit eebda78

Please sign in to comment.