Release v0.7.0

⭐ New Features

• [Feature]CURP WAL (Write-Ahead-Log) implementation：We designed a Write-Ahead-Log to persist log entries for curp.
• [Feature]Deduplication：Implements command deduplication using the exactly-once semantics from the RIFL paper.
• [Feature]: Snapshot Restore #630](#630)
• [Feature]: Interface of maintenance server #543
• [Feature]: Support run with tls [#328](#328)

🛠️ Refactor

• [Refactor]: Add a session structure to renew lock lease automatically #684
• [Refactor]: startup process #629
• [Refactor]: Refactor the GC of speculative pool #439
• [Refactor]:New command execution logic: We removed the cmd workers and related command execution logic. Now Xline will use serial execution (with batch) to prevent lock contention.
• [Refactor]: Garbage Collection: We refactored the garbage collection logic based on the dedup implementation.
• [Refactor]:Client Propose Optimization: We refactored the curp client to use gRPC stream to send a propose to the cluster. This reduces the load of the gRPC server and improves the overall performance.
• [Refactor]: Conflict Detection: We removed the conflict checked mpmc. Now Xline will use the conflict pools (speculative pool and uncommitted pool) to detect command conflicts.
• [Refactor]: Read Index: We implemented the Read Index mechanism from the Raft paper for read-only commands. Replacing the previous Read State implementation.
• [Refactor]: xline-client Refactor: We switched to a more intuitive user interface for the xline-client crate.
• [Refactor]: Various performance optimizations

🪲 Bug Fixes

• [Bug]: Implement ReadIndex #870
• [etcdapi] [Bug]: Repeated revision #848
• [Bug]: The test case curp::it server::shutdown_rpc_should_shutdown_the_cluster failed #774
• [Bug]: Xline will loss event when using watch feature #677
• [Bug]: The xlinectl won't renew the lease of the lock key #664
• [Bug]: Failed to add a new member node4 to a three-nodes xline cluster #661
• [Refactor]: read-only command during revision generation #502
• [Bug]: Read different values in execute during read state #501
• [Bug]: sync txn request does not execute in the order of child request #498
• [Bug]: Read State #497
• [Bug]: revision generation #491, #848
• [Bug]: requests in a single txn do not execute in sequence #471
• [Bug]: batch_index in raw_curp will overflow eventually #368
• [Bug]: Fix GC may break the CURP durability #159

✍️ Known Issues

During benchmark if the cluster is under high load, the cluster may sometimes not be able to make progress due to deduplication mechanism.

💥 Breaking Changes

• etcd compative APIs will now use 2-RTTs operations to prevent revision generation issues.

❤️ Contributors

We'd like to thank all the contributors who worked on this release!

• @themanforfree
• @iGxnon
• @Phoenix500526
• @bsbds

Release v0.6.1

🪲 Bug Fixes

• [Bug]: Fixed a bug causing panic during the update node process. issue #531
• [Bug]: Fixed a bug causing panic during the CI process due to reading state. issue #527
• [Bug]: Fixed a bug in the previous version where immediately shutting down the cluster after executing "member add" would cause the leader to fail to shut down properly. issue #526
• [Bug]: Fixed an issue in TXN where the conflict detection process would ignore the key of child requests. issue #470
• [Bug]: Fixed a panic issue in watch caused by closing the channel. issue #370
  • pr 576: fix: fix ce event tx logs
  • pr 556: [Fix]: kv update channel panic
• [Bug]: Changed the calculation method for the cluster version to a hash to avoid misjudgments. pr #590
• [Bug]: Fixed a bug that caused the compact operation to behave abnormally in specific scenarios. pr #570

🛠️ Refactor

• [Refactor]: Refactored the implementation of the Curp client, reducing code complexity. ：
  • pr 582: Refactor/curp client tests suits
  • pr 584: Refactor/curp client retry
  • pr 585: Refactor/replace curp client
• [Refactor]: Removed some command-related data structures from xline and xline-client. pr #469

Release v0.6.0

⭐ New Features

• [Feature]: Add membership change mechanism for CUPR consensus Protocol (Read design doc #306 for more details)
• [Feature]: Implement cluster server and client #464, #465
• [Feature]: Implement the graceful shutdown feature.
• [Feature]: Implement the xlinctl to communicate with the xline cluster. Currently, the xlinectl covers functionalities:
  • Compaction and member command: Implemented in pr #484
  • Txn, watch and lock command: Implemented in pr #428
  • Role command: Implemented in pr #427
  • User command: Implemented in pr #426
  • Snapshot and auth command: Implemented in pr #425
  • Delete and lease command: Implemented in pr #424

🪲 Bug Fixes

• [Bug]: benchmark client cannot connect to server #462
• [Bug]: remove stop in simulation tests #458
• [Bug]: execute out of order #454
• [Bug]: check the password on leader #435
• [Bug]: remove recovery of uncommitted pool #419
• [Bug]: CURP TLA+ quorum size calculation & property check #418
• [Bug]: fix propose doesn't handle SyncedError #407

🛠️ Refactor

• [Refactor]: reduce code duplication #407
• [Refactor]: Take into account the interleaving states of a request broadcast in TLA+ #429
• [Refactor]: Refine the bench client implementation #496
• [Refactor]: Simplified the error handling logic #480
• [Refactor]: Improve readability of bootstrap errors #432
• [Refactor]: Imporve command serialization in execution and after-sync #421, #422

❤️ Contributors

• @EAimTY
• @MarkGaox
• @Kikkon

Note：
Known issue: If the cluster is shut down immediately after adding a member, the leader node may not shut down properly, continuously trying to send entries to the shut-down new node. Read issue #526 for more details.

Release v0.5.0

⭐ New Features

• [Feature]: Implemented the Compact feature, which includes the following two aspects (Read issue #188 for more details about compaction design):
  • Implemented historical version compaction feature. The compaction API is compatible with the etcd interface. Resolved in pr #311
  • Implemented an automatic compaction mode, supporting both periodic Strategy and Revision Strategy. It is not enabled by default. Resolved in pr #401
• [Feature]: Implement a Rust SDK for the Xline client (xline-client crate) to fully leverage the performance of CURP protocol. Currently, the SDK covers functionalities:
  • Watch: Implemented in pr #321
  • Kv: Implemented in pr #318
  • Maintenance: Implemented in pr #323
  • Auth: Implemented in pr #320
  • Lease: Implemented in pr #319
  • Lock: Implemented in pr #322
  • Compaction: Implemented in pr #389
• [Feature]: Implement a command line tool for Xline, which is named xlinectl. Resolved in pr #348
• [Feature]: Support single node cluster #335
• [Feature]: Support multiplatform for Xline. Read #doc for more details.
• [Feature]: Support dns resolution for Xline cluster #351
• [Feature]: Support grpc health checking protocol, resolved in pr #385
• [Feature]: Add madsim simulation in Curp tests #282

🪲 Bug Fixes

• [Bug]: Madsim Curp integration tests sometimes fail #361
• [Bug]: lease may not synced in lease server #343
• [Bug]: EventListener will lose event since it's not cancellation safe. #339
• [Bug]: Resolve failing tests related to serialized size. #259

❤️ Contributors

We'd like to thank all the contributors who worked on this release!

• @liubog2008

Release v0.4.1

⭐ New Features

• [Feature]: Watch Progress Notify #309

🛠️ Refactors

• [Refactor]: sharing state between CurpServer and CurpClient #299
• [Refactor]: refactor the XlineServer #293
• [Refactor]：Refactor curp fast read implementation #270
• [Refactor]: improve the read and write logic for the RocksSnapshot #263
• [Refactor]: refactor the watch server implementation #253
• [Refactor]: refactor the dependencies of lease server #251

🪲 Bug Fixes

• [Bug]: test_kv_authorization block #291
• [Bug]: The background command workers will panic in the integration test "recovery_after_compaction" #285
• [Bug]: handle_propose in raw_cup will panic when try notify events #280
• [Bug]: Watch prev_kv #277

❤️ Contributors

We'd like to thank all the contributors who worked on this release!

• @themanforfree
• @Phoenix500526

Release v0.4.0

Features：

1. Introduce batching mechanism to improve network bandwidth utilization
2. Implement the snapshot feature for CURP consensus protocol,
3. Implement the snapshot relevant API，which is compatible with etcdctl. The rest of other APIs in etcdctl maintenance will be implemented in the future.

Fix Bugs：

1. Fix a bug that commands will execute out of order in some concurrent cases (issue #197）, resolve in the pr #195
2. Fix a bug that the gc task will panic during benchmark(issue #206), resolve in the pr #210
3. Fix a bug that the lock feature will work abnormally in some cases(issue #209), resolve in the pr #212
4. Fix a bug that some concurrent put requests will get wrong revisions (issue #209), resolve in the pr #238

Benchmark:

Since we implemented the persistence feature for xline in v0.3.0, we have re-benchmarked xline in this release. The benchmark report can be viewed in the Performance Comparison section in our README file.

Xline Release v0.3.0

Features:

• Implement a persistent storage layer to enable durability, including:
  • Implement a storage engine layer to abstract the concrete storage engine, like rocksdb,
    and enable upper layer storage function (#185, #187)
  • Enable recover logic for curp and xline (#194, #184)

Fix Bugs:

• Fix concurrent cmd order bug (#197)

Xline Release v0.2.0

Xline Release v0.2.0

Features:

• Enable Xline to boot up from the config file xline_server.conf (#145)
• Support ETCD APIs, like the lease API and the lock API (#142, #153)
• Enable the recovery mechanism in the CURP module (#146)
• Add ETCD APIs compatibility test (test report: (report)[./VALIDATION_REPORT.md])

Fix Bugs

• Fix panic in the benchmark (#123)
• Fix the issue that modifies kv pairs will fail after watching them in etcdctl (#148)

v0.1.0

v0.1.0

What is it?

Xline is a geo-distributed KV store for metadata management, which is based on the Curp protocol.

Why make it?

Existing distributed KV stores mostly adopt the Raft consensus protocol, which takes two RTTs to complete a request. When deployed in a single data center, the latency between nodes is low, so it will not have a big impact on performance. However, when deployed across data centers, the latency between nodes may be tens or hundreds of milliseconds, at which point the Raft protocol will become a performance bottleneck. The Curp protocol is designed to solve this problem. It can reduce one RTT when commands do not conflict, thus improving performance.

What does it provide?

• Etcd Compatible API
  • Kv service
  • Watch service
  • Auth service
• basic implementation of the Curp protocol
• basic Xline client (use Curp directly)
• benchmark tool

Usage

Usage doc

Note

In this release, we only provide binary files for X86_64 linux. Other platforms need to be compiled by yourself. we will add more support in the future.

Links

• GitHub: https://github.com/datenlord/Xline
• Paper of Curp: https://www.usenix.org/system/files/nsdi19-park.pdf