Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: Add performance report and help prompt #7

Merged
merged 16 commits into from
Apr 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,81 @@
# redis2asp
High-performance Aerospike proxy for the Redis protocol
### Performance Test Report
aerospike 3.x 2c4g redispike-proxy 2c4g:

`./redis-benchmark -h xxxx -p 6379 -n 2000000 -c 200 -t get`
```
WARNING: Could not fetch server CONFIG
====== GET ======
2000000 requests completed in 44.94 seconds
200 parallel clients
3 bytes payload
keep alive: 1
multi-thread: no

Latency by percentile distribution:
0.000% <= 1.975 milliseconds (cumulative count 1)
50.000% <= 3.879 milliseconds (cumulative count 1001535)
75.000% <= 4.919 milliseconds (cumulative count 1501029)
87.500% <= 5.999 milliseconds (cumulative count 1750484)
93.750% <= 7.247 milliseconds (cumulative count 1875200)
96.875% <= 8.599 milliseconds (cumulative count 1937780)
98.438% <= 10.071 milliseconds (cumulative count 1968821)
99.219% <= 12.023 milliseconds (cumulative count 1984401)
99.609% <= 13.967 milliseconds (cumulative count 1992202)
99.805% <= 16.071 milliseconds (cumulative count 1996100)
99.902% <= 19.199 milliseconds (cumulative count 1998053)
99.951% <= 22.271 milliseconds (cumulative count 1999027)
99.976% <= 24.847 milliseconds (cumulative count 1999515)
99.988% <= 26.831 milliseconds (cumulative count 1999756)
99.994% <= 30.527 milliseconds (cumulative count 1999879)
99.997% <= 31.231 milliseconds (cumulative count 1999942)
99.998% <= 31.535 milliseconds (cumulative count 1999971)
99.999% <= 31.663 milliseconds (cumulative count 1999988)
100.000% <= 31.743 milliseconds (cumulative count 1999993)
100.000% <= 31.807 milliseconds (cumulative count 1999997)
100.000% <= 31.871 milliseconds (cumulative count 1999999)
100.000% <= 31.919 milliseconds (cumulative count 2000000)
100.000% <= 31.919 milliseconds (cumulative count 2000000)

Cumulative distribution of latencies:
0.000% <= 0.103 milliseconds (cumulative count 0)
0.001% <= 2.007 milliseconds (cumulative count 10)
0.051% <= 2.103 milliseconds (cumulative count 1012)
22.646% <= 3.103 milliseconds (cumulative count 452915)
56.697% <= 4.103 milliseconds (cumulative count 1133935)
78.001% <= 5.103 milliseconds (cumulative count 1560018)
88.276% <= 6.103 milliseconds (cumulative count 1765529)
93.275% <= 7.103 milliseconds (cumulative count 1865499)
95.983% <= 8.103 milliseconds (cumulative count 1919661)
97.593% <= 9.103 milliseconds (cumulative count 1951867)
98.460% <= 10.103 milliseconds (cumulative count 1969205)
98.949% <= 11.103 milliseconds (cumulative count 1978987)
99.238% <= 12.103 milliseconds (cumulative count 1984763)
99.451% <= 13.103 milliseconds (cumulative count 1989027)
99.629% <= 14.103 milliseconds (cumulative count 1992578)
99.732% <= 15.103 milliseconds (cumulative count 1994648)
99.808% <= 16.103 milliseconds (cumulative count 1996153)
99.853% <= 17.103 milliseconds (cumulative count 1997051)
99.878% <= 18.111 milliseconds (cumulative count 1997563)
99.899% <= 19.103 milliseconds (cumulative count 1997985)
99.918% <= 20.111 milliseconds (cumulative count 1998363)
99.936% <= 21.103 milliseconds (cumulative count 1998710)
99.949% <= 22.111 milliseconds (cumulative count 1998981)
99.962% <= 23.103 milliseconds (cumulative count 1999237)
99.969% <= 24.111 milliseconds (cumulative count 1999375)
99.979% <= 25.103 milliseconds (cumulative count 1999573)
99.984% <= 26.111 milliseconds (cumulative count 1999674)
99.990% <= 27.103 milliseconds (cumulative count 1999795)
99.991% <= 28.111 milliseconds (cumulative count 1999828)
99.993% <= 29.103 milliseconds (cumulative count 1999854)
99.993% <= 30.111 milliseconds (cumulative count 1999862)
99.997% <= 31.103 milliseconds (cumulative count 1999932)
100.000% <= 32.111 milliseconds (cumulative count 2000000)

Summary:
throughput summary: 44501.80 requests per second
latency summary (msec):
avg min p50 p95 p99 max
4.337 1.968 3.879 7.679 11.255 31.919
```
34 changes: 31 additions & 3 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.example</groupId>
<artifactId>redis2asp</artifactId>
<groupId>icu.funkye</groupId>
<artifactId>redispike-proxy</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
<start-class>icu.funkye.redispike.ProxyMain</start-class>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
Expand All @@ -19,6 +20,7 @@
<log.version>1.2.9</log.version>
<p3c-pmd.version>1.3.6</p3c-pmd.version>
<maven-pmd-plugin.version>3.8</maven-pmd-plugin.version>
<asp-client.version>7.2.1</asp-client.version>
</properties>
<dependencies>
<dependency>
Expand All @@ -39,7 +41,7 @@
<dependency>
<groupId>com.aerospike</groupId>
<artifactId>aerospike-client</artifactId>
<version>7.2.1</version>
<version>${asp-client.version}</version>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
Expand Down Expand Up @@ -69,12 +71,38 @@
</dependency>
</dependencies>
<build>
<finalName>${artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.3.0</version>
<configuration>
<archive>
<manifest>
<mainClass>${start-class}</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp;
package icu.funkye.redispike;

import org.apache.commons.cli.ParseException;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,22 +14,26 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp;
package icu.funkye.redispike;

import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.Optional;
import com.aerospike.client.Host;
import com.aerospike.client.policy.ClientPolicy;
import com.alipay.remoting.ConnectionEventType;
import com.alipay.remoting.ProtocolManager;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.CommandLineParser;
import org.apache.commons.cli.DefaultParser;
import org.apache.commons.cli.HelpFormatter;
import org.apache.commons.cli.Options;
import org.apache.commons.cli.ParseException;
import org.redis2asp.common.BoltServer;
import org.redis2asp.common.CONNECTEventProcessor;
import org.redis2asp.common.DISCONNECTEventProcessor;
import org.redis2asp.factory.AeroSpikeClientFactory;
import org.redis2asp.protocol.RedisProtocol;
import icu.funkye.redispike.common.BoltServer;
import icu.funkye.redispike.common.CONNECTEventProcessor;
import icu.funkye.redispike.common.DISCONNECTEventProcessor;
import icu.funkye.redispike.factory.AeroSpikeClientFactory;
import icu.funkye.redispike.protocol.RedisProtocol;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -61,8 +65,14 @@ public void start(String... args) throws ParseException {
+ "for single record and batch commands.");
options.addOption("s", "set", true, "Set name. Use 'empty' for empty set (default: demoset)");
options.addOption("n", "namespace", true, "Namespace (default: test)");
options.addOption("u", "help", false, "Print usage.");

CommandLineParser parser = new DefaultParser();
CommandLine cl = parser.parse(options, args, false);
if (cl.hasOption("u")) {
logUsage(options);
throw new RuntimeException("Terminate after displaying usage");
}
port = Integer.parseInt(cl.getOptionValue("p", "6379"));
String host = cl.getOptionValue("th", "127.0.0.1");
int targetPort = Integer.parseInt(cl.getOptionValue("tp", "3000"));
Expand All @@ -86,8 +96,16 @@ public void start(String... args) throws ParseException {
}
}

public void shutdown() {
server.stop();
private void logUsage(Options options) {
HelpFormatter formatter = new HelpFormatter();
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
String syntax = "java -jar redispike-proxy.jar -options";
formatter.printHelp(pw, 100, syntax, "options:", options, 0, 2, null);
System.out.println(sw);
}

public void shutdown() {
Optional.ofNullable(server).ifPresent(BoltServer::stop);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp.common;
package icu.funkye.redispike.common;

import com.alipay.remoting.ConnectionEventProcessor;
import com.alipay.remoting.ConnectionEventType;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp.common;
package icu.funkye.redispike.common;

import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicBoolean;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp.common;
package icu.funkye.redispike.common;

import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp.factory;
package icu.funkye.redispike.factory;

import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
Expand Down Expand Up @@ -61,6 +61,7 @@ public static void createInstance(ClientPolicy clientPolicy, Host... hosts) {
LOCK.lock();
try {
if (client == null) {
clientPolicy.maxConnsPerNode = 65535;
clientPolicy.eventLoops = eventLoops;
client = new AerospikeClient(clientPolicy, hosts);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp.protocol;
package icu.funkye.redispike.protocol;

import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
Expand All @@ -23,10 +23,10 @@
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.DecoderException;
import org.redis2asp.protocol.request.CommandRequest;
import org.redis2asp.protocol.request.DelRequest;
import org.redis2asp.protocol.request.GetRequest;
import org.redis2asp.protocol.request.SetRequest;
import icu.funkye.redispike.protocol.request.CommandRequest;
import icu.funkye.redispike.protocol.request.DelRequest;
import icu.funkye.redispike.protocol.request.GetRequest;
import icu.funkye.redispike.protocol.request.SetRequest;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand All @@ -53,7 +53,9 @@ public void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) {

private RedisRequest<?> convert2RedisRequest(List<String> params) {
String cmd = params.get(0);
LOGGER.info("cmd: {}", params);
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("cmd: {}", params);
}
switch (cmd) {
case "get":
return new GetRequest(params.get(1));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.redis2asp.protocol;
package icu.funkye.redispike.protocol;

import java.io.Serializable;
import com.alipay.remoting.CommandEncoder;
Expand All @@ -29,7 +29,6 @@ public class RedisCommandEncoder implements CommandEncoder {

@Override
public void encode(ChannelHandlerContext ctx, Serializable msg, ByteBuf out) throws Exception {
logger.info("RedisCommandEncoder encode");
if (msg instanceof RedisResponse) {
RedisResponse<?> redisResponse = (RedisResponse) msg;
if (logger.isDebugEnabled()) {
Expand Down
Loading
Loading