Skip to content

liboffkv is a C++ library that provides a uniform interface for 3 different distributed KV-storages: etcd, ZooKeeper, Consul.

License

Apache-2.0 and 2 other licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
CC0-1.0
LICENSE-CC0
MIT
LICENSE-MIT
Notifications You must be signed in to change notification settings

offscale/liboffkv

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

liboffkv

License Travis CI Travis CI

The library is designed to provide a uniform interface for three distributed KV storages: etcd, ZooKeeper and Consul.

The services have similar but different data models, so we outlined the common features.

In our implementation, keys form a ZK-like hierarchy. Each key has a version that is int64 number greater than 0. Current version is returned with other data by the most of operations. All the operations supported are listed below.

Method Parameters Description
create key: string
value: char[]
leased: bool (=false) -- makes the key to be deleted on client disconnect
Creates the key.
Throws an exception if the key already exists or
preceding entry does not exist.
Returns: version of the newly created key.
set key: string
value: char[]
Assigns the value.
Creates the key if it doesn’t exist.
Throws an exception if preceding entry does not exist.
Returns: new version of the key.
cas key: string
value: char[]
version: int64 (=0) -- expected version of the key
Compare and set operation.
If the key does not exist and the version passed equals 0, creates it.
Throws an exception if preceding entry does not exist.
If the key exists and its version equals to specified one updates the value. Otherwise does nothing and returns 0.
Returns: new version of the key or 0.
get key: string
watch: bool (=false) -- start watching for change in value
Returns the value currently assigned to the key.
Throws an exception if the key does not exist.
If watch is true, creates WatchHandler waiting for a change in value. (see usage example below).
Returns: current value and WatchHandler.
exists key: string
watch: bool (=false) -- start watching for removal or creation of the key
Checks if the key exists.
If watch is true, creates WatchHandler waiting for a change in state of existance (see usage example below).
Returns: version of the key or 0 if it doesn't exist and WatchHandler.
get_children key: string
watch: bool (=false)
Returns a list of the key's direct children.
Throws an exception if the key does not exist.
If watch is true, creates WatchHandler waiting for any changes among the children.
Returns: list of direct children and WatchHandler.
erase key: string
version: int64 (=0)
Erases the key and all its descendants if the version given equals to the current key's version.
Does it unconditionally if version is 0.
Throws an exception if the key does not exist.
Returns: (void)
commit transaction: Transaction Commits transaction (see transactions API below).
If it was failed, throws TxnFailed with an index of the failed operation.
Returns: list of new versions of keys affected by the transaction

Transactions

Transaction is a chain of operations of 4 types: create, set, erase, check, performing atomically. Their descriptions can be found below. N.b. at the moment set has different behavior in comparison to ordinary set: when used in transaction, it does not create the key if it does not exist. Besides you cannot assign watches. Leases are still available. Transaction body is separated into two blocks: firstly you should write all required checks and then the sequence of other operations (see an example below). As a result a list of new versions for all the keys involved in set operations is returned.

Method Parameters Description
create key: string
value: char[]
leased: bool (=false)
Creates the key.
Rolls back if the key already exists or preceding entry does not exist.
set key: string
value: char[]
Set the value.
Rolls back if the key does not exist.
n.b behaviors of transaction set and ordinary set differ
erase key: string
version: int64 (=0)
Erases the key and all its descendants
if the version passed equals to the current key's version.
Does it unconditionally if the version is 0.
Rolls back if the key does not exist.
check key: string
version: int64
Checks if the given key has the specified version.
Only checks if it exists if the version is 0

Usage

#include <iostream>
#include <thread>

#include <liboffkv/liboffkv.hpp>

using namespace liboffkv;


int main()
{
    // firstly specify protocol (zk | consul | etcd) and address
    // you can also specify a prefix all the keys will start with
    auto client = open("consul://127.0.0.1:8500", "/prefix");


    // on failure methods throw exceptions (for more details see "liboffkv/client.hpp")
    try {
        int64_t initial_version = client->create("/key", "value");
        std::cout << "Key \"/prefix/key\" was created successfully! "
                  << "Its initial version is " << initial_version
                  << std::endl;
    } catch (EntryExists&) {
        // other exception types can be found in liboffkv/errors.hpp
        std::cout << "Error: key \"/prefix/key\" already exists!"
                  << std::endl;
    }


    // WATCH EXAMPLE
    auto result = client.exists("/key", true);

    // this thread erase the key after 10 seconds
    std::thread([&client]() mutable {
        std::this_thread::sleep_for(std::chrono::seconds(10));
        client->erase("/key");
    }).detach();

    // now the key exists
    assert(result);

    // wait for changes
    result.watch->wait();

    // if the waiting was completed, the existance state must be different
    assert(!client.exists("/key"));


    // TRANSACTION EXAMPLE
    // n.b. checks and other ops are separated from each other
    try {
        auto txn_result = client->commit(
            {
                // firstly list your checks
                {
                    TxnCheck("/key", 42u),
                    TxnCheck("/foo"),
                },
                // then a chain of ops that are to be performed
                // in case all checks are satisfied
                {
                    TxnErase("/key"),
                    TxnSet("/foo", "new_value"),
                }
            }
        );

        // only one set/create operation
        assert(txn_result.size() == 1 &&
               txn_result[0].kind == TxnOpResult::Kind::SET);

        std::cout << "After the transaction the new version of \"/foo\" is "
                  << txn_result[0].version << std::endl;
    } catch (TxnFailed& e) {
        // TxnFailed exception contains failed op index
        std::cout << "Transaction failed. Failed op index: "
                  << e.failed_op() << std::endl;
    }
}

Supported platforms

The library is currently tested on

  • Ubuntu 18.04

    Full support.

  • MacOS

    Full support.

  • Windows 10

    Only Consul is supported.

Dependencies

  • C++ compiler

    Currently tested compilers are

    • VS 2019
    • g++ 7.4.0
    • clang

    VS 2017 is known to fail.

  • CMake

    We suggest using cmake bundled with vcpkg.

  • vcpkg

Developer workflow

  • Install dependencies
# from vcpkg root
vcpkg install ppconsul offscale-libetcd-cpp zkpp

Installing all three packages is not required. See control flags at the next step.

  • Build tests
# from liboffkv directory
mkdir cmake-build-debug && cd $_
cmake -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_TOOLCHAIN_FILE="<replace with path to vcpkg.cmake>" \
      -DBUILD_TESTS=ON ..
cmake --build .
You can control the set of supported services with the following flags

- `-DENABLE_ZK=[ON|OFF]`
- `-DENABLE_ETCD=[ON|OFF]`
- `-DENABLE_CONSUL=[ON|OFF]`

Sometimes you may also need to specify `VCPKG_TARGET_TRIPLET`.
  • Run tests

    # from liboffkv/cmake-build-debug directory
    make test

C interface

We provide a pure C interface. It can be found in liboffkv/clib.h.

Set -DBUILD_CLIB=ON option to build the library.

liboffkv is available in other languages!!!

License

Licensed under any of:

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.

About

liboffkv is a C++ library that provides a uniform interface for 3 different distributed KV-storages: etcd, ZooKeeper, Consul.

Topics

Resources

License

Apache-2.0 and 2 other licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
CC0-1.0
LICENSE-CC0
MIT
LICENSE-MIT

Stars

Watchers

Forks