Skip to content
This repository has been archived by the owner on Oct 15, 2024. It is now read-only.

Latest commit

 

History

History

yamlcpp

README.md

  • infos = Information about the yamlcpp plugin is in keys below
  • infos/author = René Schwaiger [email protected]
  • infos/licence = BSD
  • infos/needs = base64 directoryvalue
  • infos/provides = storage/yaml
  • infos/recommends =
  • infos/placements = getstorage setstorage
  • infos/status = unittest preview unfinished concept discouraged
  • infos/metadata =
  • infos/description = This storage plugin reads and writes data in the YAML format

YAML CPP

Introduction

The YAML CPP plugin reads and writes configuration data via the yaml-cpp library.

Installation

See installation. The package is called libelektra5-yamlcpp.

Usage

You can mount this plugin via kdb mount:

sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

. To unmount the plugin use kdb umount:

sudo kdb umount /tests/yamlcpp

. The following examples show how you can store and retrieve data via yamlcpp.

# Mount yamlcpp plugin to cascading namespace `/tests/yamlcpp`
sudo kdb mount config.yaml /tests/yamlcpp yamlcpp

# Manually add a mapping to the database
echo "🔑 : 🐳"               > `kdb file user:/tests/yamlcpp`
# Retrieve the value of the manually added key
kdb get user:/tests/yamlcpp/🔑
#> 🐳

# Manually add syntactically incorrect data
echo "some key: @some  value" >> `kdb file user:/tests/yamlcpp`
kdb get "user:/tests/yamlcpp/some key"
# STDERR: .*yaml-cpp: error at line 2, column 11: unknown token.*
# ERROR: C03100
# RET: 5

# Overwrite incorrect data
echo "🔑: value" >  `kdb file user:/tests/yamlcpp`

# Add some values via `kdb set`
kdb set user:/tests/yamlcpp 🎵
kdb set user:/tests/yamlcpp/fleetwood mac
kdb set user:/tests/yamlcpp/the chain

# Retrieve the new values
kdb get user:/tests/yamlcpp
#> 🎵
kdb get user:/tests/yamlcpp/the
#> chain
kdb get user:/tests/yamlcpp/fleetwood
#> mac

# Undo modifications
kdb rm -r /tests/yamlcpp
sudo kdb umount /tests/yamlcpp

Arrays

YAML CPP provides support for Elektra’s array data type.

# Mount yamlcpp plugin to `user:/tests/yamlcpp`
sudo kdb mount config.yaml user:/tests/yamlcpp yamlcpp

# Manually add an array to the database
echo 'sunny:'       >  `kdb file user:/tests/yamlcpp`
echo '  - Charlie'  >> `kdb file user:/tests/yamlcpp`
echo '  - Dee'      >> `kdb file user:/tests/yamlcpp`

# List the array entries
kdb ls user:/tests/yamlcpp
#> user:/tests/yamlcpp/sunny
#> user:/tests/yamlcpp/sunny/#0
#> user:/tests/yamlcpp/sunny/#1

# Read an array entry
kdb get user:/tests/yamlcpp/sunny/#1
#> Dee

# You can retrieve the last index of an array by reading the metakey `array`
kdb meta-get user:/tests/yamlcpp/sunny array
# 1

# Extend the array
kdb set user:/tests/yamlcpp/sunny/#2 Dennis
kdb set user:/tests/yamlcpp/sunny/#3 Frank
kdb set user:/tests/yamlcpp/sunny/#4 Mac

# The plugin supports empty array fields
kdb set user:/tests/yamlcpp/sunny/#_10 'The Waitress'
kdb meta-get user:/tests/yamlcpp/sunny array
#> #_10
kdb get user:/tests/yamlcpp/sunny/#_9
# RET: 11

# Retrieve the last array entry
kdb get user:/tests/yamlcpp/sunny/$(kdb meta-get user:/tests/yamlcpp/sunny array)
#> The Waitress

# The plugin also supports empty arrays (arrays without any elements)
kdb meta-set user:/tests/yamlcpp/empty array ''
kdb export user:/tests/yamlcpp/empty yamlcpp
#> []

# Arrays in Elektra always require the `array` metakey.
# Otherwise the keys will be interpreted as normal key-value mappings.
kdb set user:/tests/yamlcpp/movies ""
kdb set user:/tests/yamlcpp/movies/#0 'A Silent Voice'
kdb export user:/tests/yamlcpp/movies yamlcpp
#> "#0": A Silent Voice
kdb meta-set user:/tests/yamlcpp/movies array ''
kdb export user:/tests/yamlcpp/movies yamlcpp
#> - A Silent Voice

# Undo modifications to the key database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

Nested Arrays

The plugin also supports nested arrays.

# Mount yamlcpp plugin to `user:/tests/yamlcpp`
sudo kdb mount config.yaml user:/tests/yamlcpp yamlcpp

# Add some key value pairs
kdb set user:/tests/yamlcpp/key value
kdb set user:/tests/yamlcpp/array ""
kdb set user:/tests/yamlcpp/array/#0 scalar
kdb set user:/tests/yamlcpp/array/#1/key value
kdb set user:/tests/yamlcpp/array/#1/🔑 🙈
kdb meta-set user:/tests/yamlcpp/array array '#1'

kdb ls user:/tests/yamlcpp
#> user:/tests/yamlcpp/array
#> user:/tests/yamlcpp/array/#0
#> user:/tests/yamlcpp/array/#1/key
#> user:/tests/yamlcpp/array/#1/🔑
#> user:/tests/yamlcpp/key

# Retrieve part of an array value
kdb get user:/tests/yamlcpp/array/#1/key
#> value

# Since an array saves a list of values, an array parent
# - which represent the array - does not store a value!
echo "user:/tests/yamlcpp/array: “`kdb get user:/tests/yamlcpp/array`”"
#> user:/tests/yamlcpp/array: “”

# Remove part of an array value
kdb rm user:/tests/yamlcpp/array/#1/key

kdb ls user:/tests/yamlcpp
#> user:/tests/yamlcpp/array
#> user:/tests/yamlcpp/array/#0
#> user:/tests/yamlcpp/array/#1/🔑
#> user:/tests/yamlcpp/key

# The plugin stores array keys using YAML sequences.
# Since yaml-cpp stores keys in arbitrary order -
# either `key` or `array` could be the “first” key -
# we remove `key` before we retrieve the data. This way
# we make sure that the output below will always look
# the same.
kdb rm user:/tests/yamlcpp/key
kdb file user:/tests/yamlcpp | xargs cat
#> array:
#>   - "___dirdata: "
#>   - scalar
#>   - 🔑: 🙈

# Undo modifications to the key database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

Sparse Arrays

Since Elektra allows “holes” in a key set, YAML CPP has to support small key sets that describe relatively complex data.

# Mount yamlcpp plugin
sudo kdb mount config.yaml user:/tests/yamlcpp yamlcpp

kdb set      user:/tests/yamlcpp/#0/map/#1/#0 value
kdb set      user:/tests/yamlcpp ""
kdb meta-set user:/tests/yamlcpp           array '#0'
kdb set      user:/tests/yamlcpp/#0/map ""
kdb meta-set user:/tests/yamlcpp/#0/map    array '#1'
kdb set      user:/tests/yamlcpp/#0/map/#1 ""
kdb meta-set user:/tests/yamlcpp/#0/map/#1 array '#0'
kdb file user:/tests/yamlcpp | xargs cat
#> - "___dirdata: "
#> - map:
#>     - "___dirdata: "
#>     - ~
#>     -
#>       - "___dirdata: "
#>       - value

# The plugin adds the missing array parents to the key set
kdb ls user:/tests/yamlcpp
#> user:/tests/yamlcpp
#> user:/tests/yamlcpp/#0/map
#> user:/tests/yamlcpp/#0/map/#0
#> user:/tests/yamlcpp/#0/map/#1
#> user:/tests/yamlcpp/#0/map/#1/#0

# Undo modifications to the key database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

Metadata

The plugin supports metadata. The example below shows how a basic Key including some metadata, looks inside the YAML configuration file:

key without metadata: value
key with metadata: !elektra/meta
  - value2
  - metakey: metavalue
    empty metakey:
    another metakey: another metavalue

. As we can see above the value containing metadata is marked by the tag handle !elektra/meta. The data type contains a list with two elements. The first element of this list specifies the value of the key, while the second element contains a map saving the metadata for the key. The data above represents the following key set in Elektra if we mount the file directly to the namespace user:

Name Value Metaname Metavalue
user:/key without metadata value1 — —
user:/key with metadata value2 metakey metavalue
empty metakey —
another metakey another metavalue

. The example below shows how we can read and write metadata using the yamlcpp plugin via kdb.

# Mount yamlcpp plugin to `user:/tests/yamlcpp`
sudo kdb mount config.yaml user:/tests/yamlcpp yamlcpp

# Manually add a key including metadata to the database
echo "🔑: !elektra/meta [🦄, {comment: Unicorn}]" >  `kdb file user:/tests/yamlcpp`
kdb meta-ls user:/tests/yamlcpp/🔑
#> comment
kdb meta-get user:/tests/yamlcpp/🔑 comment
#> Unicorn

# Add a new key and add some metadata to the new key
kdb set user:/tests/yamlcpp/brand new
kdb meta-set user:/tests/yamlcpp/brand comment "The Devil And God Are Raging Inside Me"
kdb meta-set user:/tests/yamlcpp/brand rationale "Because I Love It"

# Retrieve metadata
kdb meta-ls user:/tests/yamlcpp/brand
#> comment
#> rationale
kdb meta-get user:/tests/yamlcpp/brand rationale
#> Because I Love It

# Undo modifications to the key database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

We can also invoke additional plugins that use metadata like type.

sudo kdb mount config.yaml user:/tests/yamlcpp yamlcpp type
kdb set user:/tests/yamlcpp/typetest/number 21
kdb meta-set user:/tests/yamlcpp/typetest/number check/type short

kdb set user:/tests/yamlcpp/typetest/number "One"
# RET: 5
# STDERR: .*Validation Semantic.*
# ERROR: C03200

kdb get user:/tests/yamlcpp/typetest/number
#> 21

# Undo modifications to the key database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

Binary Data

YAML CPP also supports base64 encoded data via the Base64 plugin.

# Mount YAML CPP plugin at `user:/tests/binary`
sudo kdb mount test.yaml user:/tests/binary yamlcpp
# Manually add binary data
echo 'bin: !!binary aGk=' > `kdb file user:/tests/binary`

# Base 64 decodes the data `aGk=` to `hi` and stores the value in binary form.
# The command `kdb get` prints the data as hexadecimal byte values.
kdb get user:/tests/binary/bin
#> \x68\x69

# Add a string value to the database
kdb set user:/tests/binary/text mate
# Base 64 does not modify textual values
kdb get user:/tests/binary/text
#> mate

# The Base 64 plugin re-encodes binary data before YAML CPP stores the key set. Hence the
# configuration file contains the value `aGk=` even after YAML CPP wrote a new configuration.
grep -q 'bin: !.* aGk=' `kdb file user:/tests/binary`
# RET: 0

# Undo modifications to the database
kdb rm -r user:/tests/binary
sudo kdb umount user:/tests/binary

Empty

Sometimes you only want to save a key with an empty value. The commands below show that YAML CPP supports this scenario properly.

# Mount YAML CPP plugin at `user:/tests/yamlcpp`
sudo kdb mount test.yaml user:/tests/yamlcpp yamlcpp

# Check if the plugin saves empty keys correctly
kdb set user:/tests/yamlcpp/empty ""
kdb set user:/tests/yamlcpp/empty/level1/level2 ""

kdb ls user:/tests/yamlcpp/empty
#> user:/tests/yamlcpp/empty
#> user:/tests/yamlcpp/empty/level1/level2
kdb get -v user:/tests/yamlcpp/empty | grep -vq 'The key is null.'

# Undo modifications to the database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

Binary Values

Elektra saves binary data as either 0 or 1. The YAML CPP plugin supports this design decision by converting between YAML’s and Elektra’s boolean type.

# Mount YAML CPP plugin at `user:/tests/yamlcpp`
sudo kdb mount config.yaml user:/tests/yamlcpp yamlcpp
# Manually add boolean key
echo 'truth: true' > `kdb file user:/tests/yamlcpp`

kdb get user:/tests/yamlcpp/truth
#> 1

# A boolean in Elektra has the type `boolean`
kdb meta-get user:/tests/yamlcpp/truth type
#> boolean

# Add another boolean value
kdb set user:/tests/yamlcpp/success 0
kdb meta-set user:/tests/yamlcpp/success type boolean
kdb get user:/tests/yamlcpp/success
#> 0
kdb export user:/tests/yamlcpp/success yamlcpp
#> false

# Undo modifications to the database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

Dependencies

This plugin requires yaml-cpp. On a Debian based OS the package for the library is called libyaml-cpp-dev. On macOS you can install the package yaml-cpp via HomeBrew.

Limitations

Leaf Values

One of the limitations of this plugin is, that it only supports values inside leaf nodes. Let us look at an example to show what that means. The YAML file below:

root:
  subtree: 🍂
  below root: leaf
level 1:
  level 2:
    level 3: 🍁

stores all of the values (🍂, leaf and 🍁) in the leaves of the mapping. The drawing below makes this situation a little bit clearer.

Tree

The key set that this plugin creates using the data above looks like this (assuming we mount the plugin to user:/tests/yamlcpp):

Name Value
user:/tests/yamlcpp/level
user:/tests/yamlcpp/level 1/level 2
user:/tests/yamlcpp/level 1/level 2/level 3 🍁
user:/tests/yamlcpp/root
user:/tests/yamlcpp/root/below root leaf
user:/tests/yamlcpp/root/subtree 🍂

. Now why is this plugin unable to store values outside leaf nodes? For example, why can we not store a value inside user:/tests/yamlcpp/level 1/level 2? To answer this question we need to look at the YAML representation:

level 1:
  level 2:
    level 3: 🍁

. In a naive approach we might just try to add a value e.g. 🙈 right next to level 2:

level 1:
  level 2: 🙈
    level 3:  🍁

. This however would be not correct, since then the YAML node level 2 would contain both a scalar value (🙈) and a mapping ({ level 3: 🍁 }). We could solve this dilemma using a list:

level 1:
  level 2:
    - 🙈
    - level 3: 🍁

. However, if we use this approach we are not able to support Elektra’s array type properly.

Directory Values

To overcome the limitation described above, the YAML CPP plugin requires the Directory Value plugin. This plugin converts the value of a non-leaf node to a leaf node with the name ___dirdata. For example, let us assume we have the following key set:

directory      = Directory Data
directory/file = Leaf Data

. The Directory Value plugin will convert the key set in the set (write) direction to

directory            =
directory/___dirdata = Directory Data
directory/file       = Leaf Data

. Consequently the YAML plugin will store the key set as

directory:
  ___dirdata: Directory Data
  file: Leaf Data

. A user of the YAML plugin will not notice this feature unless he edits the configuration file by hand, as the following example shows:

# Mount YAML CPP plugin at `user:/tests/yamlcpp`
sudo kdb mount test.yaml user:/tests/yamlcpp yamlcpp

kdb set user:/tests/yamlcpp/directory 'Directory Data'
kdb meta-set user:/tests/yamlcpp/directory comment 'Directory Metadata'
kdb set user:/tests/yamlcpp/directory/file 'Leaf Data'

kdb ls user:/tests/yamlcpp/directory
#> user:/tests/yamlcpp/directory
#> user:/tests/yamlcpp/directory/file

kdb get user:/tests/yamlcpp/directory
#> Directory Data
kdb meta-get user:/tests/yamlcpp/directory comment
#> Directory Metadata
kdb get user:/tests/yamlcpp/directory/file
#> Leaf Data

# Undo modifications to the database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

.

Special Values

Due to the way the plugin writes data

  • first converting the key set into yaml-cpp’s Node data structure, and then
  • writing this data structure into a file,

and the way the yaml-cpp library handles writing Nodes, the plugin does currently not handle data with special meaning according to the YAML spec correctly. For example, if you use the kdb tool to save the value true in a key, then the plugin will not quote this value and you will end up with a boolean value.

# Mount plugin
sudo kdb mount test.yaml user:/tests/yamlcpp yamlcpp

kdb set user:/tests/yamlcpp/boolean true
# The following command should print a quoted YAML scalar
# (e.g. `"true"` or `'true'`).
kdb export user:/tests/yamlcpp/boolean yamlcpp
#> true

# Since the value is not quoted the YAML CPP plugin will
# correctly convert the YAML data into one of Elektra’s
# boolean values (`0` or `1`).
kdb get user:/tests/yamlcpp/boolean
#> 1

# Undo modifications to the database
kdb rm -r user:/tests/yamlcpp
sudo kdb umount user:/tests/yamlcpp

Other Limitations

  • Adding and removing keys does remove comments inside the configuration file

  • The plugin currently lacks proper type support for scalars.

  • If Elektra uses YAML CPP as default storage plugin, multiple tests of the test suite fail. However, if you mount YAML CPP at /:

    kdb mount default.yaml / yamlcpp

    all tests should work correctly. The problem here is that Elektra does not load additional required plugins (infos/needs) for a default storage plugin.