Skip to content

Commit

Permalink
Add kalray plugin to configure DPU
Browse files Browse the repository at this point in the history
This plugin allows to create raid, logical volume store and logical
volumes on the Kalray DPU. It also allows the deletion of a volume.
Currently restrications are due to the Kalray DPU. Next generation of
the DPU will allow more volumes and won't be restricted on volume name.
See the REAMDE for details.

Signed-off-by: Guillaume <[email protected]>
  • Loading branch information
gthvn1 committed Oct 17, 2023
1 parent e9415e0 commit 3760d59
Show file tree
Hide file tree
Showing 2 changed files with 249 additions and 0 deletions.
98 changes: 98 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,104 @@ $ xe host-call-plugin host-uuid=<uuid> plugin=hyperthreading.py fn=get_hyperthre
true
```

## XCP-ng Kalray DPU

A xapi plugin to get information about raids, logical volume store (LVS) and
devices that are present on the Kalray DPU. It also allow the management of
logical volumes (LV): creation and deletion. Parameters depends of the name of
the command some are always available:
- *username*: username to use to connect to DPU (required)
- *password*: password to connect to DPU (required)
- *server*: IP of the server for configuring the DPU (default: localhost)
- *port*: Port to use (default: 8080)
- *timeout*: timeout in second (default: 60.0)

### Command details

- Currently the Kalray DPU is still in developpement and there are some
restrictions:
- To be able to expose virtual functions the Kalray poller expects
specific names for the logical volume store and for the volume. It
depends of the configuration used in `/etc/kalray/0000:XX:00.1.conf`.
- By default the logical volume store name **must be** `lvs`.
- By default the volume must start with `volume_`.
- With the current DPU only four virtual functions (and so only four NVMe
disks) can be created and so you can only use the following name:
- `volume_09`
- `volume_10`
- `volume_11`
- `volume_12`
- Only volumes can be deleted.

#### Block devices

##### Get the list of devices on the Kalray DPU
```
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=get_devices \
args:username=<username> args:password=<password>
{"status": "ok", "output": [{"name": "HotInNvmeWDS500AFY0-22050C800415n1", "aliases": [], "product_name": "NVMe disk", "block_size": 512, "num_blocks": 976773168, "uuid": "e8238fa6-bf53-0001-001b-448b45afa6a7", "assigned_rate_limits": {"rw_ios_per_sec": 0, "rw_mbytes_per_sec": 0, "r_mbytes_per_sec": 0, "w_mbytes_per_sec": 0}, "claimed": false, "zoned": false, "supported_io_types": {"read": true, "write": true, "unmap": true, "write_zeroes": true, "flush": true, "reset": true, "nvme_admin": true, "nvme_io": true}, "driver_specific": {"nvme": [{"pci_address": "0000:00:00.0", "trid": {"trtype": "PCIe", "traddr": "0000:00:00.0"}, "ctrlr_data": {"cntlid": 8224, "vendor_id": "0x15b7", "model_number": "WDS500G1X0E-00AFY0", "serial_number": "22050C800415", "firmware_revision": "614900WD", "subnqn": "nqn.2018-01.com.wdc:nguid:E8238FA6BF53-0001-001B448B45AFA6A7", "oacs": {"security": 1, "format": 1, "firmware": 1, "ns_manage": 0}, "multi_ctrlr": false, "ana_reporting": false}, "vs": {"nvme_version": "1.4"}, "ns_data": {"id": 1, "can_share": false}, "security": {"opal": false}}], "mp_policy": "active_passive"}}]}
```

#### RAID

##### Create a raid on the Kalray DPU
- Supported RAID are raid0, raid1 and raid10
```
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=raid_create \
args:username=<username> args:password=<password> \
args:base_bdevs=HotInNvmeWDS500AFY0-22050C800415n1,HotInNvmeWDS500AFY0-22050C800378n1 \
args:raid_name=raid0 \
args:raid_level=raid0
true
```

##### Get the list of raids on the Kalray DPU
```
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=get_raids \
args:username=<username> args:password=<password>
["raid0"]
```

#### Logical Volume Store (LVS)
##### Create an LVS on the Kalray DPU
```
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=lvs_create \
args:username=<username> args:password=<password> \
args:lvs_name=lvs \
args:bdev_name=raid0
"6fb90332-56e4-4d03-aa6a-f858a2c2ca97"
```

##### Get the list of LVS on the Kalray DPU
```
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=get_lvs \
args:username=<username> args:password=<password>
[{"uuid": "6fb90332-56e4-4d03-aa6a-f858a2c2ca97", "name": "lvs", "passive": false, "base_bdev": "raid0", "total_data_clusters": 29804, "free_clusters": 29772, "block_size": 512, "cluster_size": 33554432}]
```

#### Logical Volume (LVOL)
##### Create a new logical volume
```
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=lvol_create \
args:username=<username> args:password=<password> \
args:lvol_name=volume_09 \
args:lvol_size_mb=1048576 \
args:lvs_name=lvs
"6c84b44c-a61b-41a4-8b19-32ab643b57d9"
```

##### Delete a logical volume
- The name of the volume to be deleted is not the same than the one used to
create it. You need to prepend the name of the logical volume store as shown
in the example:

```
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=lvol_delete \
args:username=<username> args:password=<password> \
args:lvol_name=lvs/volume_09
true
```

## Tests

To run the plugins' unit tests you'll need to install `pytest`, `pyfakefs` and `mock`.
Expand Down
151 changes: 151 additions & 0 deletions SOURCES/etc/xapi.d/plugins/kalray_dpu.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
#!/usr/bin/python3
"""XAPI plugin to manage Kalray DPU."""

import json
import XenAPIPlugin # pylint: disable=import-error

from kalray.acs.spdk import rpc # pylint: disable=import-error
from xcpngutils import error_wrapped

class KalrayCmd:
"""Describe a command to be ran on the Kalray DPU."""

def __init__(self, rpc_name: str, updates: dict):
self.server = 'localhost'
self.port = 8080
self.username = None
self.password = None
self.timeout = 60.0
self.rpc_name = rpc_name
self.rpc_params = {} # will be updated using add_rpc_params

for k, v in updates.items():
if hasattr(self, k):
setattr(self, k, v)

# Check that username & password are well set
if self.username is None:
raise XenAPIPlugin.Failure("-1", ["'username' is required"])
if self.password is None:
raise XenAPIPlugin.Failure("-1", ["'password' is required"])

def add_rpc_params(self, key, value):
"""Adds a parameter that will be passed to the RPC."""
self.rpc_params[key] = value

def call_rpc(self):
"""Do the RPC call."""
try:
client = rpc.client.HTTPJSONRPCClient(
self.server,
self.port,
self.timeout,
self.username,
self.password,
log_level="ERROR")
message = client.call(self.rpc_name, self.rpc_params)
except rpc.client.JSONRPCException as exc:
raise XenAPIPlugin.Failure("-1", [exc.message])

return json.dumps(message)

@error_wrapped
def get_devices(_session, args):
"""Get the list of devices available on the Kalray DPU."""
kc = KalrayCmd("bdev_get_bdevs", args)
return kc.call_rpc()

@error_wrapped
def get_raids(_session, args):
"""Get the list of raids available on the Kalray DPU."""
kc = KalrayCmd("bdev_raid_get_bdevs", args)
kc.add_rpc_params("category", "all")
return kc.call_rpc()

@error_wrapped
def get_lvs(_session, args):
"""Get the list of logical volume stores available on the Kalray DPU."""
kc = KalrayCmd("bdev_lvol_get_lvstores", args)
return kc.call_rpc()

@error_wrapped
def raid_create(_session, args):
"""Create a raid."""
kc = KalrayCmd("bdev_raid_create", args)
try:
raid_name = args["raid_name"]
raid_level = args["raid_level"]
base_bdevs = args["base_bdevs"]
except KeyError as msg:
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])

# Sanity checks
if raid_level not in ["raid0", "raid1", "raid10"]:
raise XenAPIPlugin.Failure("-1", ["Only raid0, raid1 and raid10 are supported"])
# TODO: check that bdevs are "NVMe disk"

kc.add_rpc_params("name", raid_name)
kc.add_rpc_params("raid_level", raid_level)
kc.add_rpc_params("base_bdevs", base_bdevs.split(','))
kc.add_rpc_params("strip_size_kb", 128)
kc.add_rpc_params("persist", True)
kc.add_rpc_params("split_dp", True)
return kc.call_rpc()

@error_wrapped
def lvs_create(_session, args):
"""Create a logical volume store."""
kc = KalrayCmd("bdev_lvol_lvs_createtore", args)
try:
lvs_name = args["lvs_name"]
bdev_name = args["bdev_name"]
except KeyError as msg:
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])

kc.add_rpc_params("lvs_name", lvs_name)
kc.add_rpc_params("bdev_name", bdev_name)

return kc.call_rpc()

@error_wrapped
def lvol_create(_session, args):
"""Create a new lvol on the Kalray DPU."""
kc = KalrayCmd("bdev_lvol_create", args)

try:
lvol_name = args["lvol_name"]
lvol_size = int(args["lvol_size_mb"])
lvs_name = args["lvs_name"]
except KeyError as msg:
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])
except ValueError as msg:
raise XenAPIPlugin.Failure("-1", [f"Wrong size: {msg}"])

kc.add_rpc_params("lvol_name", lvol_name)
kc.add_rpc_params("size", lvol_size)
kc.add_rpc_params("lvs_name", lvs_name)
return kc.call_rpc()

@error_wrapped
def lvol_delete(_session, args):
"""Delete the lvol passed as parameter on the Kalray DPU if exists."""
kc = KalrayCmd("bdev_lvol_delete", args)

try:
lvol_name = args["lvol_name"]
except KeyError as msg:
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])

kc.add_rpc_params("name", lvol_name)
return kc.call_rpc()

if __name__ == "__main__":
XenAPIPlugin.dispatch({
"get_devices": get_devices,
"get_raids": get_raids,
"get_lvs": get_lvs,
"raid_create": raid_create,
"lvs_create": lvs_create,
"lvol_create": lvol_create,
"lvol_delete": lvol_delete,
})

0 comments on commit 3760d59

Please sign in to comment.