1.1.0

Update vstart version to 18.2.2 (fixes #493)
Improve gateway metadata and add connection information metrics fixes #414)
Prometheus version string was changed, need to remove the prefix "v" (fixes #492)
Pass anagrp and not request.anagrpid to SPDK (fixes #484)
Add load balancing group to gateway's info CLI (fixes #471)
Remove model number from "subsystem list" output (fixes #468)
Change serial number to use Ceph and not SPDK (fixes #469)
Use "Ceph" instead of "SPDK" in model number (fixes #470)
Increase the number of log file backup directories and add the limit to the conf file (fixes #478)
Make sure not to include "_" in OMAP key components (fixes #426)
recycling of namespaces, bdevs and clusters for blocklisted ana group
ceph monitor integration (for more details see ceph/ceph#54671)
Add a help message with command options in CLI (fixes #466)
Add a test for creating and deleting namespaces with two gateways (fixes #463)
Show max namespaces in sybsystem list (fixes #430)
Image size should be 64 bits (fixes #456)
Add a test for "connection list" (fixes #448)
Fix format string in the "host list" command (fixes #452)
Make sure we always hold the RPC lock in find_namespace_and_bdev_name() (fixes #444)
Allocate cluster context per ANA group
Add exceptions full information to the log (fixes #438)
Use a second level of subparsers in the CLI (fixes #432)
Make sure to use the correct NSID when creating namespaces in update() (fixes #435)
Do not use the same RBD image for two namespaces (fixes #415)
Add HA state transitions tests
List all listeners of a subsystem, from all gateways (fixes #408)
monitor client integration (for more details see ceph/ceph#54671)
Add timestamp and code location to console log messages (fixes #418)
Add messaged about the location of the log files (fixes #404)

1.0.0

• Version 1.0.0 base version for backward compatibility
• Make sure listener add works in multi-gateway after subsystem and listeners re-creation Fixes #403
• docker-compose: support v1 and v2
• Clean protobuf from unused features, hard code TCP for transport type and only allow IP addr family Fixes #397
• Use dev Ceph binaries for build Fixes #395
• Create RBD image if needed when adding a namespace Fixes #386
• docker,ci,make: add Ceph-Dashboard support and other improvementsDisplay SPDK commit hash in the gateway's log Fixes #383
• Check subsystem and host NQN validity before passing them to SPDK. Fixes #364
• Add log messages for huge pages count. Fixes #378
• When we get a request to create a listener with a foreign gateway name in an update() do not display an error. Fixes #375
• Add SPDK version to "gw info" command. Fixes #356
• Squeeze the width of the namespace list table. Fixes #368
• Only send discovery service async notifications for subsystem and listener updates. Fixes #324
• Make port a mandatory argument for delete listener command. Fixes #360
• Make the "host" parameter mandatory in "host add" and "host del" CLI commands. Fixes #361
• Add prometheus exporter to the gateway
• Save NVMeOF log in files under /var/log/ceph. Fixes #317

v0.0.7

This version is not compatible with any older OMAP state files.
New CLI (the old CLI is obsolete and there is no backward compatibility to versions prior to 0.0.7)
Update Ceph vstart version to 18.2.1
Fix #346 listeners enable-ha state is lost when the gateway restarts
Fix #253 Use enum and not string for some GRPC fields
Fix #340 get_subsystems fails with "namespace" has no field named "anagrpid"
Fix #335 Make port parameter not required for create and delete listener CLI commands
Fix #332 Add a get_gateway_info CLI command
Fix #327 Fix OMAP lock test
Fix #284 create_subsystem should fail if --enable-ha=true, and --ana-reporting=false

v0.0.6

1. Fix an issue in DiscoveryController - Host reconnects when Python discovery controller sends Async event – #323
2. grpc: return structured subsystems_info.
3. Disable AVX512 in SPDK on github builds - #320
4. Add a bdev resize CLI command - Fixes - #226
5. Disable VPCLMULQDQ when we build inside github - #311
6. Lock Omap file before changing it to protect against multi gateway corruption In case the Omap file was changed, read it again before trying to change it - #56
7. Block the usage of the same serial number for two subsystems - #148
8. Do not duplicate the code to construct resource keys - #275
9. Do not allow using the discovery NQN in the CLI commands - #299
10. Make sure to convert enum key to proper case before fetching the enum value - #300
11. Handle IPv6 parameters in NVMe CLI - #245 #247
12. Create the SPDK socket under /var/run/ceph/ and not /var/tmp - #216
13. Display the actual Ceph version in the NVMEof log - #293
14. When comparing state values in update() make sure both values are the same type - #241
15. Avoid code duplication when connecting to Rados - #289
16. Change phrasing of Ceph versin log message Fixes - #287
17. Update Ceph to version 18.2.0 - #285
18. discovery: use rados_id to connect to ceph cluster.
19. Write SPDK and discovery service PIDs to the log. Display conf file earlier - #281
20. Add a container name for the gateway so we could use it in demo.mk - #265
21. Do not allow duplicate add_host and create_listener commands - #144
22. Get local state and not OMAP's for ANA processing - #276
23. Make sure OMAP's get_state() reads the entire file, even if it has more than 1024 keys - #263, #266
24. Don't call get_subsystems from within delete_bdev() - #260
25. set anagroup per added namespace, set ana_report_enabled per create subsystem, added test
26. Added a new CLI command to get gw info. The command will return - #338:

• cli_version // the CLI image version
• gateway_version
• gateway_name
• gateway_group
• gateway_addr
• gateway_port

v0.0.5

1. Add version and GIT repo information to log, and also print conf file contents to the log - #267
2. Add a lock for GRPC calls to prevent a race from causing a corruption. The race is between loading the omap values, and getting CLI calls in parallel #258
3. Change default discovery service to using the new python discovery service code #249
4. Add nvmeof version and config file path to the log messages #256
5. Add initial support for ANA and HA in GW CLI - #264

v0.0.4

1. spdk cherry pick: bdev/rbd: Do not submit IOs through thread sending.
2. Set default block size in create_bdev CLI to 512 instead of 4K. Fixes #224
3. Issue #28 Use bdev_rbd_register_cluster
4. control/discovery: add discovery controller
5. fix(grpc): throw error on wrong gateway listener
6. Generate coredump when no KA to the SPDK
7. Increase the huge pages count to 4GB so we could support 256 connections.
8. Disable SPDK discovery controller on GW initialization Fix multi-gateway test

v0.0.3

What's Changed

• deploy: ceph-nvmeof GW can be deployed using ceph orchestrator - by @adk3798 ceph/ceph#50423 and ceph/ceph#52691
• Add nvme connect, and IO testing to CI - by @baum #189
• create_bdev: pass ceph user id to SPDK - by @baum #194
• Add controller id range to ceph-nvmeof.conf and pass to gateway. This allows using the partioning of controller id range between gateways, to be done later by Ceph-adm - by @gbregman #186
• bump SPDK version to 23.01.1 LTS - by @baum #183
• Add a persistent UUID in create_bdev, generated UUID should be persistent - by @gbregman #181

Full Changelog: v0.0.2...v0.0.3

Container Images

nvmeof - quay.io/ceph/nvmeof:0.0.3
nvmof-cli - quay.io/ceph/nvmeof-cli:0.0.3
spdk - quay.io/ceph/spdk:23.01.1

v0.0.2

What's Changed

• build: create stand-alone packages by @epuertat in #169
• control/state: add name to rados.Rados object initialization by @adk3798 in #166
• security: update gRPC by @epuertat in #171
• release: v0.0.2 by @epuertat in #173

New Contributors

• @adk3798 made their first contribution in #166

Full Changelog: v0.0.1...v0.0.2

v0.0.1

First Ceph NVMe-oF Pre-Release

This is the first pre-release of the Ceph NVMe-oF project.

Containerized Deployment

Containers for this pre-release are available at quay.io:

• https://quay.io/ceph/nvmeof:0.0.1
  • Note: This container only runs on x86-64 platforms with RDSEED CPU support (known issue: #153).
• https://quay.io/ceph/nvmeof-cli:0.0.1

Bare-metal Deployment

See attached packages below (RPM and Python packages).

What's Changed

• Add SPDK submodule. by @sdpeters in #1
• Initial commit for the NVMeoF gateway to Ceph management daemon by @anitashekar in #3
• nvme_gw_server: allow to change rpc socket address by @trociny in #4
• nvme_gw: allow to list and delete bdevs and subsystems by @trociny in #5
• nvme_gw_server: allow to pass additional options to spdk target by @trociny in #7
• nvme_gw_server: nvmf_tgt process cleanup by @trociny in #6
• nvme_gw_server: rename tgt_cmd_args config to tgt_cmd_extra_args by @trociny in #8
• Add target save and restore functionality by @sskaur in #9
• spdk: update to 22.01 by @trociny in #15
• nvme_gw_server: fixup for spdk update to 22.01 by @trociny in #21
• nvme_gw_server: use dedicated rpc client for pings by @trociny in #19
• nvme_gw_server: do not redirect tgt stderr/out to pipe by @trociny in #16
• nvme_gw: implicitly create transports by @trociny in #18
• nvme_gw_server: use only one grpc worker thread by @trociny in #25
• don't require address when adding/removing listener by @trociny in #17
• Restructure (split gRPC and server code, better naming, Makefile) by @sskaur in #31
• Make bdev name optional by @sskaur in #53
• README.md: fix the example for add_host command by @CongMinYin in #54
• fix spdk rpc import path by @sskaur in #86
• spdk: update to 23.01 by @idryomov in #79
• server: increase the default for conn_retries by @trociny in #81
• ceph-nvmeof.conf: comment out transports option by @idryomov in #93
• Initial multi-gateway support by @sskaur in #29
• server: provide a default for tgt_cmd_extra_args option by @idryomov in #95
• server: Don't mute exceptions by @epuertat in #98
• spdk: add --no-huge support by @epuertat in #103
• docs: Create CONTRIBUTING.md by @epuertat in #125
• build: check PR deps by @epuertat in #139
• control/state: change order in listener omap key by @trociny in #138
• feat(control): allow specifying max_namespaces on subsystem creation by @rahullepakshi in #150
• build: containerize deployment by @epuertat in #90

New Contributors

• @sdpeters made their first contribution in #1
• @anitashekar made their first contribution in #3
• @trociny made their first contribution in #4
• @sskaur made their first contribution in #9
• @CongMinYin made their first contribution in #54
• @idryomov made their first contribution in #79
• @epuertat made their first contribution in #98
• @rahullepakshi made their first contribution in #150

Full Changelog: https://github.com/ceph/ceph-nvmeof/commits/v0.0.1