Simple Java API to run performance tests, using JMeter as engine, in an VCS (versioning control system) and programmers friendly way.
If you use maven, just include following dependency:
<dependency>
<groupId>us.abstracta.jmeter</groupId>
<projectId>jmeter-java-dsl</projectId>
<version>0.1</version>
</dependency>Here is a simple example test in JUnit 5+ with 2 threads/users iterating 10 times each to send HTTP POST requests with a JSON body to http://my.service:
import static org.assertj.core.api.Assertions.assertThat;
import static us.abstracta.jmeter.javadsl.JmeterDsl.*;
import java.time.Duration;
import org.eclipse.jetty.http.MimeTypes.Type;
import org.junit.jupiter.api.Test;
import us.abstracta.jmeter.javadsl.TestPlanStats;
public class PerformanceTest {
@Test
public void testPerformance() throws IOException {
TestPlanStats stats = testPlan(
threadGroup(2, 10,
httpSampler("http://my.service")
.post("{\"name\": \"test\"}", Type.APPLICATION_JSON)
),
//this is just to log details of each request stats
jtlWriter("test.jtl")
).run();
assertThat(stats.overall().elapsedTimePercentile99()).isLessThan(Duration.ofSeconds(5));
}
}This example also uses AssertJ for assertions, but you can use whatever assertion library you chose.
More examples can be found in tests
Tip 1: Since JMeter uses log4j2, if you want to control logging level or output, you can use something similar to the tests included log4j2.xml.
Tip 2: When working with multiple samplers in a test plan, specify their names to easily check their respective statistics.
There are many tools to script performance/load tests, being JMeter and Gatling the most popular ones.
JMeter is great for people with no programming knowledge since it provides a graphical interface to create test plans and run them. Additionally, it is the most popular tool (with a lot of supporting tools built on it) and has a big amount of supported protocols and plugins that makes it very versatile.
But, JMeter has some problems as well: sometimes might be slow to create test plans in JMeter GUI, and you can't get the full picture of the test plan unless you dig in every tree node to check its properties. Furthermore, it doesn't provide a simple programmer friendly API (you can check here for an example on how to run jmeter programmatically without jmeter-java-dsl), nor a VCS friendly format (too verbose and hard to review). For example, for the same test plan previously showed with jmeter-java-dsl, in JMeter you would need a JMX file like this, and even then, it wouldn't be as simple to do assertions on collected statistics as in provided example.
Gatling does provide a simple API and a VCS friendly format, but requires scala knowledge and environment. Additionally, it doesn't provide as rich environment as JMeter (protocol support, plugins, tools) and requires learning a new framework for testing (if you already use JMeter, which is the most popular tool).
Taurus is another open-source tool that allows specifying tests in a VCS friendly yaml syntax, and provides additional features like pass/fail criteria and easier CI/CD integration. But this tool requires a python environment, in addition to the java environment. Additionally, there is no built-in GUI or IDE auto-completion support, which makes it harder to discover and learn the actual syntax. Finally, Taurus syntax only supports a subset of the features JMeter provides, which reduces scope usage.
Finally, ruby-dsl is also an opensource library which allows specifying and run in ruby custom dsl JMeter test plans. This is the most similar tool to jmeter-java-dsl, but it requires ruby (in addition to java environment) with the additional performance impact, does not follow same naming and structure convention as JMeter, and lacks of debugging integration with JMeter execution engine.
jmeter-java-dsl tries to get the best of these tools by providing a simple java API with VCS friendly format to run JMeter tests, taking advantage of all JMeter benefits and knowledge also providing many of the benefits of Gatling scripting. As shown in previous example, it can be easily executed with JUnit, modularized in code and easily integrated in any CI/CD pipeline. Additionally, it makes it easy to debug the execution of test plans with usual IDE debugger tools. Finally, as with most Java libraries, you can use it not only in a Java project, but also in projects of most JVM languages (like kotlin, scala, groovy, etc).
Here is a table with summary of main pros and cons of each tool:
| Tool | Pros | Cons |
|---|---|---|
| JMeter |
|
|
| Gatling |
|
|
| Taurus |
|
|
| ruby-dsl |
|
|
| jmeter-java-dsl |
|
|
Currently, the project only covers the basic, but most used, features when implementing JMeter performance tests. The idea is to evaluate if the community (you) is interested in using the library, and if so, implement new features as the community request them, covering at some point most of JMeter (and plugins) features. In order to accomplish this, we need you to please create an issue for any particular feature or need that you have.
We would also really appreciate pull requests. Check the CONTRIBUTING guide for an explanation of main library components and how can you extend the library.
Finally, if you like this project, please give it a star, that helps the project to be more visible and gain relevance.