Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
4a555ea
Bump Pulsar 2.9.1 and Kafka client 3.0.0 (#238)
codelipenghui Jan 20, 2022
3b793a2
Set flushEntrylogBytes (#240)
codelipenghui Jan 20, 2022
ddd4b90
Fix missing dash (#244)
EladLeev Feb 14, 2022
8c116f5
Use the latest apache pulsar grafana dashboard (#252)
Demogorgon314 Mar 7, 2022
44293d5
Load balance topics on RabbitMQ nodes (#239)
cbornet Mar 10, 2022
73081cf
Create a benchmark driver for KoP (https://github.com/streamnative/ko…
BewareMyPower Mar 10, 2022
380acf7
[Pulsar] Fix available zone not configured and clean up the terraform…
BewareMyPower Mar 10, 2022
89dce6d
Support deploying protocol handlers for Pulsar (#251)
BewareMyPower Mar 10, 2022
1701fca
[Pulsar] Add ansible playbook to restart broker (#257)
BewareMyPower Mar 29, 2022
800e1af
[Pravega Driver] [Draft] Update dependencies for Pravega 0.12 (#259)
AndreyKoltsov1997 Apr 7, 2022
0d02d56
Fix RabbitMQ docs (#255) (#256)
HScarb Apr 7, 2022
9a8254d
Bump jackson-databind from 2.9.10.8 to 2.12.6.1 (#261)
dependabot[bot] Apr 7, 2022
64cc01d
Test warmup option (#260)
novosibman Apr 7, 2022
64eff25
Fixes #247 Fix Coordinated Omission (#248)
franz1981 Apr 7, 2022
0209bd1
Fix copyright (#243) (#263)
HScarb Apr 8, 2022
214e33e
Revert "[Pravega Driver] [Draft] Update dependencies for Pravega 0.12…
vongosling Apr 8, 2022
2d5e568
[ISSUE #265]Fix com/fasterxml/jackson/annotation/JsonKey: NoClassDefF…
zhangjidi2016 Apr 12, 2022
c1200d6
[ISSUE #268]rocketmq benchmark support acl (#269)
zhangjidi2016 May 1, 2022
4a02411
Switch to RHEL-8 since RHEL-7.4 doesn't work anymore on EC2
merlimat Jul 3, 2022
7ec1817
Simplified JVM GC options
merlimat Jul 3, 2022
48c31db
Disabling protocol handlers in ansible
merlimat Jul 3, 2022
0aa74e3
Upgrade to Pulsar 2.10.1
merlimat Jul 3, 2022
72778d8
[Pulsar] Specify the NIC speed when deploying brokers
merlimat Jul 4, 2022
3241915
Fixed benchmark worker config to work with Java 17
merlimat Jul 4, 2022
5b8209f
Use bigger VMs for ZooKeeper deployment
merlimat Jul 4, 2022
7a16bc5
Fixed BK memory settings
merlimat Jul 4, 2022
cca7698
Fixing infinite loop waiting for consumers to be ready (#276)
franvila Jul 28, 2022
31855cd
Fix license check and modify configurations for multi topics test (#282)
codelipenghui Aug 26, 2022
df0d9d3
[issue #275] fix waiting for consumers to be ready (#277)
franvila Aug 26, 2022
8294b0a
Bump jetty-server in /benchmark-framework (#274)
dependabot[bot] Aug 26, 2022
94178fa
Change zookeeper node type to i3en.2xlarge for multi-topics test (#287)
codelipenghui Sep 5, 2022
7c55e60
Revert "Bump jetty-server in /benchmark-framework (#274)" (#289)
nahguam Sep 8, 2022
53108b5
Create topics in batches to minimise requests and retry failures (#288)
Sep 12, 2022
bb2cdaf
Update RMQ versions (#294)
teabot Sep 14, 2022
4d6f53f
[RabbitMQ] Introduce AZ var, default to us-west-2a (#293)
teabot Sep 14, 2022
882a5bc
Improve visibility and stability of benchmark execution (#295)
Sep 14, 2022
d923ce8
Add ansible tags to kafka (#297)
Sep 15, 2022
f812235
Improve Kafka infrastructure stability (#298)
Sep 15, 2022
59cd951
[Improvement] Consolidate project dependencies (#296)
teabot Sep 15, 2022
13de8a6
Add a shutdownhook to stop all workers (#299)
Sep 15, 2022
d493354
Remove Kafka reset option (#300)
Sep 15, 2022
2f55db6
Remove catch IOException. Follow up to #299 (#301)
Sep 15, 2022
44371da
[Feature] Command Line Utility to create workloads from a template (#…
teabot Sep 16, 2022
3d04126
Add backlog drain timings to the log (#303)
Sep 16, 2022
146886c
Move shutdown hook (#304)
Sep 16, 2022
5892855
Added license header to tool/pom.xml
merlimat Sep 16, 2022
2b763f6
Fixed NATS driver to use JetStream API
merlimat Sep 16, 2022
7b9f069
Cleanup logging (#305)
Sep 20, 2022
76513dc
Expose setters (#306)
teabot Sep 20, 2022
81cfdb9
[CI] Introduce working CI (#307)
teabot Sep 20, 2022
bf6f7e2
Remove unused (#308)
teabot Sep 20, 2022
6ec5ba8
[CI] Include code quality plugins (#309)
teabot Sep 20, 2022
8c4fabb
[RabbitMQ] Clean-up deploy (#310)
teabot Sep 20, 2022
edd9868
[Bugfix] Temporarily skip checks (#311)
teabot Sep 20, 2022
e7ca4e7
[CI] Regularise licenses (#312)
teabot Sep 20, 2022
7ebe40b
[CI] Enable spotbugs in the build (#316)
teabot Sep 21, 2022
545ded1
[Bugfix] Do not allocate throughput to workers with no producers (#320)
teabot Sep 22, 2022
5b39969
[Feature] Collect producer error stats (#317)
teabot Sep 22, 2022
519ae68
Fix spotless with markdown (#318)
Sep 22, 2022
c6f77f3
Allow checkstyle to be suppressed in specific cases (#319)
Sep 22, 2022
f093dc3
Allow spaces in empty for iterator (#321)
Sep 22, 2022
5e8f2ae
[RabbitMQ] Expose management port (#323)
teabot Sep 27, 2022
a1b5bc3
Use quorum queues (#325)
teabot Sep 27, 2022
929c4c3
[Unit Tests] Basic tests 1 (#326)
teabot Sep 28, 2022
d5141e1
Reformat code (#324)
Sep 29, 2022
3d88d7b
Keep running for testDurationMinutes after a backlog drain (#328)
Sep 30, 2022
8f7d5d6
[Refactor] Break up ensemble to enable testing (#327)
teabot Sep 30, 2022
be60e24
[RabbitMQ] Prometheus & Grafana (#330)
teabot Oct 6, 2022
c4ad972
Proportional min backlog (#335)
teabot Oct 7, 2022
174e907
Fix adjustments of the the max sustainable rate (#336)
Oct 7, 2022
3abea16
[RabbitMQ] Queue type configuration option (#331)
teabot Oct 7, 2022
07895af
Allow deployment of benchmark code in isolation (#339)
teabot Oct 18, 2022
eb7d284
→ AWS, +Prometheus +Grafana, +doc, versions++ (#340)
teabot Oct 18, 2022
d7cc6c3
[NATS] Deploy in AWS with observability (#342)
teabot Oct 18, 2022
7175eca
[fix][artemis] Fix Artemis asynchronous send and acknowledgement conf…
lhotari Oct 24, 2022
910bd7c
Use unique names (#344)
teabot Oct 24, 2022
5967b3f
Use unique names (#345)
teabot Oct 24, 2022
00502be
New max sustainable rate implementation (#329)
Oct 24, 2022
1b48fa7
Bump commons-text from 1.9 to 1.10.0 in /tool (#346)
dependabot[bot] Oct 24, 2022
276693b
Increase buffer sizes (#349)
teabot Oct 26, 2022
1c3eece
NATS config changes (#348)
teabot Oct 26, 2022
d707b87
Allow deployment of benchmark code in isolation (#347)
teabot Oct 26, 2022
dc4da71
[Feature] Support shared subscriptions (#353)
teabot Oct 27, 2022
620419d
Convenience ansible (#350)
teabot Jan 9, 2023
1066759
Create and share consumer (#356)
teabot Jan 9, 2023
7f910c4
Stream type (#357)
teabot Jan 9, 2023
9cb6764
Set partition handling behaviour (#358)
teabot Jan 9, 2023
d04724a
Clients require all broker address to fail over (#359)
teabot Jan 9, 2023
c3e271d
Resilient resource creation (#362)
teabot Jan 25, 2023
ed82764
[Improvement] Pulsar 2.10.2 (#354)
teabot Jan 25, 2023
defa46f
Pulsar 2.11.0
merlimat Mar 1, 2023
5eebc5b
Bump testng from 7.6.1 to 7.7.0 in /driver-kop (#366)
dependabot[bot] Mar 2, 2023
6a947eb
Fix failed drive-kop tests (#387)
BewareMyPower Aug 11, 2023
31215cc
Bump log4j from 2.17.1 to 2.20.0 (#374)
skykistler Aug 11, 2023
1c681e9
Add maven exclusion for netty transitive dependency in artemis (#376)
skykistler Aug 11, 2023
b3a9044
[NATS] Extend to support in-memory stream (#370)
Aug 28, 2023
f2a4e8c
Bumped Jedis to v5.0.0 - fixed breaking changes from v3.7.0 (#390)
filipecosta90 Sep 7, 2023
0a68a61
Issue 391: Update Pravega library versions and deployment script (#392)
RaulGracia Sep 26, 2023
71b6ff2
update rocketmq engine
Nov 16, 2023
d0c0277
fix codestyle
Nov 16, 2023
f423781
Fixed for 'Failed to execute goal com.diffplug.spotless:spotless-mave…
StevenLuMT Dec 14, 2023
4eb2327
Grow buffer if histogram can't fit (#398)
travisdowns Dec 20, 2023
13785ca
Bump com.rabbitmq:amqp-client from 4.8.0 to 5.18.0 in /driver-rabbitm…
dependabot[bot] Dec 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
45 changes: 45 additions & 0 deletions .github/workflows/pr-build-and-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Build and test

on:
pull_request:
branches:
- master
push:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
distribution: temurin
java-version: 17

- name: Build and Verify
run: mvn --no-transfer-progress --batch-mode verify

- name: package surefire test results
if: failure()
run: |
rm -rf test-results
mkdir test-results
find . -type d -name "*surefire*" -exec cp --parents -R {} test-results/ \;
zip -r test-results.zip test-results
- uses: actions/upload-artifact@v3
name: upload test-results
if: failure()
with:
name: test-results
path: test-results.zip
17 changes: 0 additions & 17 deletions .travis.yml

This file was deleted.

42 changes: 33 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,47 @@
# OpenMessaging Benchmark Framework
[![Build Status](https://app.travis-ci.com/openmessaging/benchmark.svg?branch=master)](https://app.travis-ci.com/openmessaging/benchmark)
[![Total alerts](https://img.shields.io/lgtm/alerts/g/openmessaging/benchmark.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/openmessaging/benchmark/alerts/)
[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/openmessaging/benchmark.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/openmessaging/benchmark/context:python)
[![Language grade: Java](https://img.shields.io/lgtm/grade/java/g/openmessaging/benchmark.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/openmessaging/benchmark/context:java)
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

[![Build](https://github.com/openmessaging/benchmark/actions/workflows/pr-build-and-test.yml/badge.svg)](https://github.com/openmessaging/benchmark/actions/workflows/pr-build-and-test.yml)
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

**Notice:** We do not consider or plan to release any unilateral test results based on this standard. For reference, you can purchase server tests on the cloud by yourself.
**Notice:** We do not consider or plan to release any unilateral test results based on this standard. For reference, you can purchase server tests on the cloud by yourself.

This repository houses user-friendly, cloud-ready benchmarking suites for the following messaging platforms:

* [Apache ActiveMQ Artemis](https://activemq.apache.org/components/artemis/)
* [Apache Bookkeeper](https://bookkeeper.apache.org)
* [Apache Kafka](https://kafka.apache.org)
* [Apache Pulsar](https://pulsar.apache.org)
* [Apache RocketMQ](https://rocketmq.apache.org)
* Generic [JMS](https://javaee.github.io/jms-spec/)
* [KoP (Kafka-on-Pulsar)](https://github.com/streamnative/kop)
* [NATS JetStream](https://docs.nats.io/nats-concepts/jetstream)
* [NATS Streaming (STAN)](https://docs.nats.io/legacy/stan/intro)
* [NSQ](https://nsq.io)
* [Pravega](https://pravega.io/)
* [RabbitMQ](https://www.rabbitmq.com/)
* [Apache Pulsar](https://pulsar.apache.org)
* [NATS Streaming](https://nats.io/)
* [Redis](https://redis.com/)
* [Pravega](https://pravega.io/)

> More details could be found at the [official documentation](http://openmessaging.cloud/docs/benchmarks/).

## Build

Requirements:

* JDK 8
* Maven 3.8.6+

Common build actions:

| Action | Command |
|---------------------------------|------------------------------------------|
| Full build and test | `mvn clean verify` |
| Skip tests | `mvn clean verify -DskipTests` |
| Skip Jacoco test coverage check | `mvn clean verify -Djacoco.skip` |
| Skip Checkstyle standards check | `mvn clean verify -Dcheckstyle.skip` |
| Skip Spotless formatting check | `mvn clean verify -Dspotless.check.skip` |
| Format code | `mvn spotless:apply` |
| Generate license headers | `mvn license:format` |

## License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0
Loading