openAMDC is an open source, high-performance, in-memory, key-value database. It is compatible with all the commands, protocol and data structures that were originally available in Redis, making it feasible to utilize openAMDC as a direct substitute for Redis. The architecture of openAMDC is as follows:
The following are the main enhancements of openAMDC in comparison to Redis:
-
Multi-threaded architecture: openAMDC is a multi-process architecture, which means that it can be run on multiple cores. When openAMDC starts, it will initialize multiple worker threads. In each worker thread, an event loop will be started for listen events, and the SO_REUSEPORT option is set for each listener to enable the kernel load balancing of network connections, allowing multiple threads to bind the same port. It is equivalent to that each thread has a completed queue of the listen socket in the queue, avoiding the resource contention of the shared listen socket and enhancing the concurrent throughput. The kernel evenly distributes new connections to each thread that has enabled the SO_REUSEPORT attribute through the load balancing algorithm.
-
Data swap architecture: Redis employs memory as the storage medium, which exhibits excellent performance and low latency, but its memory capacity often turns into a bottleneck, and the memory price is relatively high, causing a high usage cost of Redis. openAMDC extends the data swap function on the basis of the multi-threaded architecture, and uses RocksDB as the back-end storage engine to achieve the multi-level storage of openAMDC data to reduce the comprehensive usage cost.
worker-threads is a configuration item of openAMDC, which is used to set the number of working threads of openAMDC. By increasing the number of working threads, the concurrent processing ability of the openAMDC server can be improved, thereby enhancing the overall performance.
openamdc-server --h 127.0.0.1 --p 6379 --worker-threads 8 --save ""
-
openamdc-benchmark is a benchmarking tool for openAMDC. Remember to enable the threads option for testing.
openamdc-benchmark -h 127.0.0.1 -p 6379 -n 5000 -c 50 --threads 8 -
memtier_benchmark is a command line utility developed by RedisLabs for load generation and bechmarking NoSQL key-value databases.
memtier_benchmark -s 127.0.0.1 -p 6379 -t 8 -c 50 -n 5000 --hide-histogram --distinct-client-seed --command="set __key__ __data__" --key-prefix="kv_" --key-minimum=1 --key-maximum=10000 -R -d 128
Some test results using memtier_benchmark:
Some request latency results under 8 threads:
| 8 threads | Ops/sec | Avg Latency | p50 Latency | p99 Latency | p99.9 Latency | KB/sec |
|---|---|---|---|---|---|---|
| openAMDC | 543274.68 | 0.84819 | 0.823 | 1.335 | 4.511 | 88541.95 |
| redis | 199626.36 | 2.01135 | 1.703 | 5.151 | 7.935 | 32534.75 |
openAMDC can be compiled and used on Linux, OSX, OpenBSD, NetBSD, FreeBSD. We support big endian and little endian architectures, and both 32 bit and 64 bit systems.
It may compile on Solaris derived systems (for instance SmartOS) but our support for this platform is best effort and openAMDC is not guaranteed to work as well as in Linux, OSX, and *BSD.
It is as simple as:
% make
To build with TLS support, you'll need OpenSSL development libraries (e.g. libssl-dev on Debian/Ubuntu) and run:
% make BUILD_TLS=yes
To build with systemd support, you'll need systemd development libraries (such as libsystemd-dev on Debian/Ubuntu or systemd-devel on CentOS) and run:
% make USE_SYSTEMD=yes
To append a suffix to openAMDC program names, use:
% make PROG_SUFFIX="-alt"
You can build a 32 bit openAMDC binary using:
% make 32bit
After building openAMDC, it is a good idea to test it using:
% make test
If TLS is built, running the tests with TLS enabled (you will need tcl-tls
installed):
% ./utils/gen-test-certs.sh
% ./runtest --tls
openAMDC has some dependencies which are included in the deps directory.
make does not automatically rebuild dependencies even if something in
the source code of dependencies changes.
When you update the source code with git pull or when code inside the
dependencies tree is modified in any other way, make sure to use the following
command in order to really clean everything and rebuild from scratch:
make distclean
This will clean: jemalloc, lua, hiredis, linenoise.
Also if you force certain build options like 32bit target, no C compiler
optimizations (for debugging purposes), and other similar build time options,
those options are cached indefinitely until you issue a make distclean
command.
If after building openAMDC with a 32 bit target you need to rebuild it
with a 64 bit target, or the other way around, you need to perform a
make distclean in the root directory of the openAMDC distribution.
In case of build errors when trying to build a 32 bit binary of openAMDC, try the following steps:
- Install the package libc6-dev-i386 (also try g++-multilib).
- Try using the following command line instead of
make 32bit:make CFLAGS="-m32 -march=native" LDFLAGS="-m32"
Selecting a non-default memory allocator when building openAMDC is done by
setting the MALLOC environment variable. openAMDC is compiled and linked
against libc malloc by default, with the exception of jemalloc being the
default on Linux systems. This default was picked because jemalloc has proven
to have fewer fragmentation problems than libc malloc.
To force compiling against libc malloc, use:
% make MALLOC=libc
To compile against jemalloc on Mac OS X systems, use:
% make MALLOC=jemalloc
By default, openAMDC will build using the POSIX clock_gettime function as the monotonic clock source. On most modern systems, the internal processor clock can be used to improve performance. Cautions can be found here: http://oliveryang.net/2015/09/pitfalls-of-TSC-usage/
To build with support for the processor's internal instruction clock, use:
% make CFLAGS="-DUSE_PROCESSOR_CLOCK"
openAMDC will build with a user-friendly colorized output by default. If you want to see a more verbose output, use the following:
% make V=1
To run openAMDC with the default configuration, just type:
% cd src
% ./openamdc-server
If you want to provide your openamdc.conf, you have to run it using an additional parameter (the path of the configuration file):
% cd src
% ./openamdc-server /path/to/openamdc.conf
It is possible to alter the OpenAmdc configuration by passing parameters directly as options using the command line. Examples:
% ./openamdc-server --port 9999 --replicaof 127.0.0.1 6379
% ./openamdc-server /etc/openamdc/6379.conf --loglevel debug
All the options in redis.conf are also supported as options using the command line, with exactly the same name.
Please consult the TLS.md file for more information on how to use openAMDC with TLS.
You can use openamdc-cli to play with openAMDC. Start a openamdc-server instance, then in another terminal try the following:
% cd src
% ./openamdc-cli
openamdc> ping
PONG
openamdc> set foo bar
OK
openamdc> get foo
"bar"
openamdc> incr mycounter
(integer) 1
openamdc> incr mycounter
(integer) 2
openamdc>
In order to install openAMDC binaries into /usr/local/bin, just use:
% make install
You can use make PREFIX=/some/other/directory install if you wish to use a
different destination.
Make install will just install binaries in your system, but will not configure init scripts and configuration files in the appropriate place. This is not needed if you just want to play a bit with openAMDC, but if you are installing it the proper way for a production system, we have a script that does this for Ubuntu and Debian systems:
% cd utils
% ./install_server.sh
Note: install_server.sh will not work on Mac OSX; it is built for Linux only.
The script will ask you a few questions and will setup everything you need to run openAMDC properly as a background daemon that will start again on system reboots.
You'll be able to stop and start openAMDC using the script named
/etc/init.d/openamdc_<portnumber>, for instance /etc/init.d/openamdc_6379.
Note: Contribute code to the openAMDC project by sending a pull request via Gitee, you agree to release your code under the terms of the MulanPSLv2 license that you can find in the COPYING file included in the openAMDC source distribution. You will include MulanPSLv2 license in the COPYING file within each source file that you contribute.
-
If it is a major feature or a semantical change, create an issue at Gitee with the description of, exactly, what you want to accomplish and why. Use cases are important for features to be accepted.
-
If in step 1 you get an acknowledgment from the project leaders, use the following procedure to submit a patch;
a. Fork openAMADC on gitee
b. Create a topic branch (git checkout -b my_branch)
c. Push to your branch (git push origin my_branch)
d. Initiate a pull request on gitee (https://help.gitee.com/base/pullrequest/Fork+Pull)
e. Done :) -
For minor fixes just open a pull request on Gitee.
- Data swap architecture, openAMDC stores hot data and cold data in memory and RocksDB respectively, expands the storage capacity of openAMDC, and supports the automatic exchange and balance of memory data and SSD data.
- Compatibility Upgrade from redis 6.2 to redis 7.2.