Getting Started with Jmeter
Jmeter: http://jmeter.apache.org
NTT-PF-Lab OpenStack JMeter Home: https://github.com/ntt-pf-lab/openstack-jmeter
JMeter is a 100% Java application and should run conveniently on any system that has a compliant Java implementation.
Below are the steps to install Jmeter on Ubuntu Linux server (Tested on Ubuntu Server 11.04/Natty) for Command Line (CLI) usage, since the test suite contains shell scripts as part of a supporting tools repository.
- Install Java - JMeter requires a fully compliant JVM 1.5 or higher.
(Sun Java JRE 1.6 or OpenJre1.6 or higher on Linux.)
Download the jmeter binary
$ wget http://apache.mirrors.tds.net//jmeter/binaries/apache-jmeter-2.6.tgz
Latest Release - Apache Jmeter 2.11
Download URL: http://jmeter.apache.org/download_jmeter.cgi
- Unzip the downloaded Jmeter distribution to the desired path. (Eg. /opt/JMETER_INSTALL_DIR)
- Go to “bin” directory
- Run Jmeter file
*Windows : double click jmeter.bat
*Mac : double click ApacheJmeter.jar
*Linux: double click jmeter.sh OR run ./jmeter from the command line
$ apache-jmeter-2.6/jmeter -n -l <samples_log_file> -t <test_plan.jmx> -p <properties_file> -q <additional_properties_file>
where -n: Run in non-gui mode
-l: Path to log the samples logs. Eg: samples.log or samples.jtl
-t: Run the following test plan
-p: (Optional) Custom user.properties or keystone.properties to load.
-q: (Optional) Additional properties file containing project or test specific properties to load.
By default, only the jmeter/bin/keystone.properties and jmeter/bin/user.properties are loaded. If
the test needs an additional set of properties to be loaded, the file should be specified using the -q option.
Eg: $ ~/apache-jmeter-2.6/jmeter -n -l samples.log -t servers.jmx -q servers.properties
###1. Jmeter Plugins
This library of plugins is required to add better graphs to the standard Jmeter package.
Download: http://code.google.com/p/jmeter-plugins/downloads/list
Install: Copy the following file into JMETER_INSTALL_DIR/lib/ext folder.
- JMeterPlugins.jar
Restart JMeter and see jp@gc-prefixed plugins in menus in GUI mode.
###2. JMeterPluginsCMD Command Line Tool for Graphing (part of Jmeter Plugins source package) The zip file of the JMeter Plugins also contains a command line utility for generating graphs out of JTL files. The tool can generate PNG, CSV or both.
Install: Copy the following three files into JMETER_INSTALL_DIR/lib/ext:
- CMDRunner.jar
- JMeterPluginsCMD.bat (shell wrapper for Windows)
- JMeterPluginsCMD.sh (shell wrapper for Linux)
Restart Jmeter.
###3. ShellExecutor.jar
This extension to Jmeter is required for execution of Shell scripts from Jmeter. This utility also has the capability to capture status code and output of the script executed.
Download: https://docs.google.com/open?id=0BwufSlaYow2GWWhwWkg5N0tUUEd5ZEhTd1lCUm9yQQ
Install: Copy the following file to JMETER_INSTALL_DIR/lib/ext.
- shellexecutor.jar
(shellexecutor.jar written by Tushar Patil)
GUI-mode View
###1. Clone the repo:
$ git clone [email protected]:ntt-pf-lab/openstack-jmeter.git
###2. Navigate to openstack-jmeter performance directory:
$ cd openstack-jmeter/performance
###3. Configuation a) Configure Jmeter properties - Jmeter has 2 main property files to configure the application:
- jmeter.properties - The properties of Jmeter can be modified by implementing changes in the jmeter.properties
in the bin/ folder of jmeter. - user.properties - Additional jmeter properties can be configured in this file, also located in the bin/ folder of jmeter.
Note: openstack-jmeter test suite does not require to modify the above property files as part of configuration.
b) Configure Test Plan properties - Different test plans in the openstack-jmeter suite require custom property files where
properties are defined. Although these properties can also be defined in the above two properties files, it is better to define such test plan specific properties in different .properties files.
Jmeter command line option -q <additional_property_file(s)> can be used to pass these property files while running test
plans. For eg:
$ apache-jmeter-2.6/jmeter -n -q servers.properties -t servers_testplan.jmx
Currently, the properties for various test plans have been defined in a commmon file that resides in
'performance/common/properties/perftest.properties'. Following are the required properties:
# Common test properties
script_path= # Absolute path to scripts directory in openstack-jmeter
test_data_dir= # Absolute path to test_data directory in keystone folder of openstack-jmeter
reports_dir= # Absolute path of user defined reports directory. Note that this folder should have permissions for write access by Jenkins user.
nova_log_path= # Absolute path of syslog log file that logs nova service logs.
cmd_runner_dir= # Absolute path of CMDRunner.jar in lib/ext of jmeter source directory.
# Thread Properties for various Test plans
keystone_test_data_group.threads= # Number of threads
keystone_test_data_group.loops= # Number of times to execute the test
keystone_test_data_group.rampup= # Ramp-up period: How long to take to "ramp-up" to the full number of threads chosen
create_networks_group.threads=
create_networks_group.loops=
create_networks_group.rampup=
create_server_group.threads=
create_server_group.loop=
create_server_group.rampup=
# Keystone Test plan properties
keystone_server_ip=
keystone_admin_port=
keystone_service_port=
admin_auth_token=
users_per_thread=
tenants_per_thread=
create_user_uri=
create_tenant_uri=
create_token_uri=
users_test_data_file=
tenants_test_data_file=
# Servers Test plan properties
flavor_ref=
image_ref=
nova_api_server_ip=
nova_api_port=
request_uri=
instances_count=
snapshots_count=
# Networks Test plan properties
label=jm_net
bridge_interface=
bridge=
# Nova CLI management access properties
host= # I.P address of host where nova-manage is installed
username= # User name for SSH access
password= # Password of the user for SSH access
nova_manage_path= # Directory where nova-manage utility is located. Eg. /opt/stack/nova-manage
IMPORTANT: Do not forget to set the reports_dir parameter and give it write permissions.
c) Configure openstack-jmeter runner script -
Configure the following in openstack-jmeter/performance/run_tests.sh
- JMETER_DIR= # Path to Jmeter source base directory (Eg: /opt/jmeter/apache-jmeter-2.6)
- PERF_BASE_DIR= # Path to openstack-jmeter test suite 'performance' directory (Eg: /home/tester/openstack-jmeter/performance)
d) Configure settings for nova_api_perf_analyzer -
Configure the following in **openstack-jmeter/performance/scripts/nova_api_perf_analyzer.conf **
- result_file_prefix= # Prefix to give the results csv file of nova-api logs (Eg: nova_api)
- result_file_dir= # Base Directory to store the CSV files (Eg: /home/tester/openstack-jmeter/performance/reports)
Some configurations need to be made on the nova-api server to enable openstack-jmeter to capture server logs for metrics calculations.
###1. Enable debug logging:
Download openstack-utils from the following URL and follow install procedure.
NTT PF LAB openstack-utils
###2. Set up NFS, configure Syslog daemon:
If openstack-jmeter is running from another host than the Nova servers then NFS should be setup for access to syslogs.
Set up NFS server (on Openstack server):
1. $ sudo apt-get install nfs-kernel-server
2. $ mkdir ~/openstack_logs
3. $ sudo mkdir -p /export/openstack_logs
4. $ sudo mount --bind ~/openstack_logs /export/openstack_logs
5. Check "NEED_SVCGSSD=no" in /etc/default/nfs-kernel-server
6. In "/etc/default/nfs-common" check
NEED_IDMAPD=no
NEED_GSSD=no
7. In "/etc/idmapd.conf" check
[Mapping]
Nobody-User = nobody
Nobody-Group = nogroup
8. Add the following two lines to /etc/exports
/export 10.2.3.0/24(rw,fsid=0,insecure,no_subtree_check,no_root_squash,async)
/export/openstack_logs 10.2.3.0/24(rw,nohide,insecure,no_subtree_check,no_root_squash,async)
9. $ sudo /etc/init.d/nfs-kernel-server restart
Set up NFS client (on Jenkins server):
1. sudo apt-get install nfs-common
2. In "/etc/default/nfs-common" check:
NEED_IDMAPD=no
NEED_GSSD=no
3. sudo mount -t nfs4 -o proto=tcp,port=2049 10.2.3.164:/ /mnt
4. If error: mount.nfs4: No such device
$ sudo modprobe nfs
5. Retry #3.
6. df –h
Syslog configuration:
Configure the syslog daemon on the central syslog server, so that the Nova service logs are generated in the NFS share:
1. Update /etc/rsyslog.d/50-default.conf, replace the 'user.*' entry with the following:
user.* /export/openstack_logs/user.log
2. Restart syslog daemon:
$ sudo service rsyslog restart
It is preferred to run openstack-jmeter and Jenkins CI on the same host.
Follow the steps below to configure Jenkins job for openstack-jmeter Performance Testing and reports generation.
###1. Create a new Job
###2. Configuration: Add shell command to run openstack-jmeter tests
###3. Configuration: Add shell commands to configure latest reports symlink for Reports viewing and storing past reports
Save the configuration.
Click Build Now on the Jenkins job and then view Console Output to monitor execution. This should invoke the run_tests.sh script that calls the .jmx test plans.
After the job is executed, two reports are published:
1. Performance Metrics Report
The following reports are generated by JMeter Plugins:
- Aggregate Report (csv only)
- Response Codes Per Second (csv and png)
- Latencies Over Time (csv and png)
- Perf Mon (csv and png - Only if Jmeter server agent is running on OpenStack servers and configured)
- Response Times over Time (csv and png)
- Response Times Distribution (csv and png)
- Hits Per Second (csv and png)
- Response Times Percentiles (csv and png)
2. Log Analysis Report
This report generates API-PER-SERVICE response times and service level summary reports after parsing nova syslogs. PNG as well as CSV files are generated for Service Level summary reports and only CSV tablular data is generated for individual service response times.
- Response Time trend Example - Response Time across services for Snapshot API:
- Average API Response Time Summary - Per Instance Type Example:
- Response Time Trend across services Example - With Instance Count, Per instance type:
All test run logs of jmeter application are stored in 'jmeter.log'. If samples file is configured on the command line, then
sampler data (xml format) is written into that file.
On Jenkins, the jmeter.log path is generally on /var/lib/jenkins/jobs/(name_of_project)/workspace/jmeter.log
Jenkins Console Output also contains certain information that is written to stdout during test execution.