Releases: GoogleCloudPlatform/PerfKitBenchmarker
Releases · GoogleCloudPlatform/PerfKitBenchmarker
v1.7.0: New features:
* Add ProfitBricks provider (thanks @tjb1019, GH-1005) * Add YCSB JDBC benchmark (thanks @k1xme, GH-1049) * Add support for st1 and sc1 disks on EBS (GH-1067) Breaking changes: * Change default gce_migrate_on_maintenance to True (GH-1069) Enhancements: * Update OpenStack provider to use OpenStacks CLI instead of python library (thanks @meteorfox, GH-942) * Improve side-by-side tool (GH-1034) * Specify flags to metadata (GH-1047) * Allow Java version for Java benchmarks to be selected via command line (GH-1051) * Add numjobs runtime option to fio_benchmark (GH-1066) * Add netperf histogram support (GH-1072) * Add multi-vm support for object_storage_service_benchmark (GH-1074) * Add support for custom s3 endpoints for object_storage_service_benchmark (GH-1078) * Add user-defined metadata for vms (GH-1080) Bugfixes and maintenance updates: * Refactor object_storage_service_benchmark (GH-1023) * Support overriding ycsb_client_vms in benchmark spec (thanks @k1xme, GH-1042) * Update YCSB to 0.9.0 and support latest logging format (thanks @k1xme, GH-1043) * Fix zones flag (GH-1044) * Update Bigtable to v2 API (thanks @tswast, GH-1045) * Update maven version to 3.3.9 (GH-1046) * Start background workload in prepare stage and stop in cleanup stage (GH-1048) * Minor fixes in flag description (GH-1052) * Only upload files needed for API testing script for object_storage_service_benchmark (GH-1053) * Fix redis_ycsb benchmark to use multiple processes (GH-1062, GH-1068, GH-1070) * Fix bug with custom machine types configs (GH-1077) * Fix in documentation of ParseTimeCommandResult (thanks @skorgu, GH-1079) * Replace Pandas dependency with Numpy (GH-1081) * Turn off OpenBlas threading, improving HPCC on large vms (GH-1084) * Cleanup object_storage_serivce benchmark (GH-1085) * Cleanup aerospike_ycsb benchmarks (GH-1086)
v1.6.0: New features:
New features:
- Apache Spark benchmark (GH-1014,GH-1011)
- Cloudsuite dataserving benchmark (thanks @ustiugov, GH-930)
Enhancements:
- Optionally publish VM hostnames in metadata (GH-1020)
- Control which netperf benchmarks are run via flag (GH-1029)
- Netperf benchmark reports min and max latency (GH-1013)
- Add multichase to the google benchmark set (GH-1018)
- Update Cloudsuite web serving benchmark (thanks @ivonindza, @GH-998)
- Object storage updates (GH-987)
- Repeat Run stage (GH-1032)
Bugfixes and maintenance updates:
- Fix path to cassandra-cli (thanks @AdamIsrael, GH-1006)
- Update ycsb version; affects cloud_bigtable_ycsb_benchmark, hbase_ycsb
(GH-1021, GH-1025, GH-1031) - Use latest gsutil version (GH-1012)
- Remove duplicate requirements files (GH-975)
- Update openblas version (GH-1003)
- Background workload refactor (GH-1033)
v1.5.0
1.5.0
New features:
- Add cloudsuite graph analytics benchmark (thanks @mdrumond; GH-986)
- Add initial implementation of multichase benchmark (GH-977)
- Add cloudsuite media streaming benchmark (thanks @ivonindza; GH-993)
- Add analysis functions for processing multistream object storage benchmark data (GH-905)
- Add CloudSuite Data-caching benchmark (thanks @neo-apz; GH-970)
- Add ability to bundle PKB into a python-executable zip file (GH-971)
- Add os_type for Juju Mixin (thanks @AdamIsrael; GH-764)
- Add CloudSuite in-memory analytics benchmark (thanks @ivonindza; GH-934)
Breaking changes:
- Replace the old CloudSuite Web Search benchmark with the docker version (thanks @javpicorel; GH-931)
- Update Aerospike version (GH-978)
Enhancements:
- Add --fio_blocksize option (GH-943)
- Update speccpu2006 benchmark to support iso file (GH-944)
- Allow --run_stage to specify multiple stages (GH-935)
- Add flag to control iperf timeout (GH-994)
Bugfixes and maintenance updates:
- Fix cloud bigtable and hbase (GH-1001)
- Install openssl for the object storage benchmark (GH-1000)
- Fix bug in MongoDB YCSB benchmark (GH-999)
- Change import_util.LoadModulesForPath to also load packages (GH-983)
- Add 'ap-northeast-2' to the S3 regions table (GH-990)
- Change check-lint.sh to invoke flake8 via tox (GH-979)
- Change GetLastRunUri to handle the new temp directory structure (GH-985)
- Support S3 Signature Version 4 (GH-984)
- Add back support for python-gflags version 2. (GH-973)
- Define units.Unit.ne (GH-976)
- Reorder some PKB initialization steps (GH-965)
- Move to gflags version 3.0.4 (GH-969)
- Modify semantics of UnitsParser's convertible_to parameter (GH-968)
- Add percent (%) as a recognized pint unit (GH-964)
- Improve Azure CLI version check (GH-967, GH-958)
- Don't request SSH verbose output when --log_level=debug (GH-962)
- Move per-provider package requirement files (GH-961)
- Move pint unit registry code into a separate module (GH-960)
- Only check python package requirements if requirements.txt exists (GH-959)
- Changed LoadProvider to accept un-lowered cloud provider name (GH-957)
- Change bg tests to patch and verify call counts for each VM separately (GH-954)
- Make BenchmarkSpec.vms order consistent across runs (GH-953)
- Fix os type related issues and bugs (GH-950, GH-955, GH-952, GH-951, GH-949)
- Call CheckPrerequisites earlier (GH-947)
- Add percentiles option to PercentilesCalculator (GH-910)
- Fix publisher bug when disk_size is None (GH-946)
- Handle unset PYTHONPATH in tox.ini (GH-941)
- Add a helper function to BenchmarkConfigSpec to redirect flags (GH-936)
- Add support for interrupting child threads created by RunThreaded (GH-926)
- Fix bugs and improve the Cloud Bigtable benchmark (GH-937, GH-933, GH-932)
- Add a helpful error message if a UnitsParser parses a unitless value (GH-963)
v1.4.0: Release 1.4.0
New features: * openstack: Add support for optional floating ip pool (thanks @meteorfox, GH-861) * openstack: Use Keystone session for handling authentication (thanks @meteorfox, GH-870) * Support object storage classes in object storage service benchmarks (GH-895) * Add Object Size Distributions in object storage service benchmarks (GH-888) * Add MultiStreamThroughput benchmark to object storage benchmarks (GH-840) * Adds a SPEC SFS 2014 benchmark which runs against GlusterFS (GH-876) * gce_virtual_machine: Support user provided instance metadata (GH-859) Improvements and fixes: * openstack: Wait until VM deletion has been completed (thanks @meteorfox, GH-904) * openstack: Fix floating IP allocation and deallocation (thanks @meteorfox, GH-862) * rackspace: Fix missing flags bug (thanks @meteorfox, GH-903) * Allow user to specify a flag when they run object storage benchmark on GCP, and default that flag to the latest known working version. (GH-925) * Update mechanism used to get iperf server process id. (GH-921) * Rename variables and improve documentation of BaseOsMixin.PullFile. (GH-923) * Fix WindowsMixin._GetNumCpus when more than one Win32_processor exists. (GH-920) * Add exception types to except clauses. (GH-893) * Add Flag for List Consistency Iterations (GH-889) * Add unit tests for scripts (GH-882) * Add disk type examples to README.md (GH-871) * Copy-edit the README (GH-877) * Turn off selinux for mongodb. (thanks @akrzos, GH-867) * Use temp files for Popen stdout and stderr in IssueCommand. (GH-878)
v1.3.0: External contributions:
* Add RHEL based virtual machines to OpenStack provider. (thanks @akrzos; GH-858) * Change rackspace provider information to use official CLI (thanks @meteorfox; GH-844) * Add rackspace requirements (thanks @meteorfox; GH-805) New features: * Support flags in YAML format. (GH-857) * Check version of required packages at runtime (GH-834) * User-specified multiregion for GCS benchmarking (GH-845) * Support metadata for gcp instances (GH-859) Bugfixes and maintences updates: * Change rackspace's AllowPort to enable UDP (thanks @meteorfox; GH-805) * Allow most recent verison gcs-oauth2-boto-plugin (GH-849) * Require Pint >= 0.7 (GH-850) * Update PIP (GH-842) * Fix windows log message (GH-832) * Properly Pickle Pint Quantities (GH-830) * os_type added to boot benchmark metadata (GH-826) * Better handle Azure timeouts (GH-825) * Better handling of AWS integration tests. (GH-869, GH-868)
v1.2.0: Release 1.2.0
New features: * Introduce Object Sizes (GH-808). * Add ListDecoder for verifying a config option that expects list values (GH-807). Enhancements: * Bump HBase to 1.0.3 (GH-822). * Change MockFlags to be more like the real FlagValues (GH-812). * Rename test_flag_util.py and add FlagDictSubstitution tests (GH-811). * Create BenchmarkConfigSpec to aggregate benchmark input checking (GH-810). * Remove flag proxy objects (GH-802). Bugfixes and maintenance updates: * Fix sample iperf sample (GH-801). * FixIntDecoder and FloatDecoder behavior when min=0 or max=0 (GH-800).
v1.1.0: Release 1.1.0
External contributions: * Add additional percentile samples to mysql_service benchmark and others (thanks @zbjornson; GH-729). * Add Mesos-Marathon as a provider (thanks @mateusz-blaszkowski; GH-679, GH-731). * Use only available floating-ips for OpenStack (thanks @meteorfox; GH-733). * Add OpenStack CLI-based support for object_storage_service benchmark (thanks @meteorfox; GH-738). * Make printed benchmark results more readable (thanks @wonderfly; GH-747). * Clean up formatting, grammar, and markdown in README.md (thanks @mbrukman; GH-748). * Add links to projects and licenses in README.md (thanks @mbrukman; GH-763). * Fix a bug that caused --helpxml to not work (thanks @jldiaz; GH-782). * Make OpenStack VM username configurable with a new --openstack_image_username flag (thanks @lwatta; GH-788). New features: * Allow users to specify a storage region for the object_storage_service benchmark with a new --object_storage_region flag (GH-609). * Add support for running a background CPU workload while executing a benchmark (GH-715, GH-762). * Allow creating GCE custom VMs easily from command-line flags (GH-727). * Add the --gce_network_name to allow GCE users to use pre-existing networks (GH-746). * Publish a PTRANS sample from the hpcc benchmark (GH-785). * Add support for running a background network workload while executing a benchmark (GH-786). * Add a CSV sample output format (GH-791). Enhancements: * Trim blocks while formatting and mounting disks (GH-692). * Raise an error when no requests succeed in redis_benchmark (GH-702). * Update the Google benchmark set (GH-705). * Add input-checking to disk specs (GH-675, GH-779). * Let disks on Windows machines return Windows DeviceIds (GH-743). * Speed up Cassandra cluster boot by setting "auto_bootstrap" to false (GH-751). * Defer package update for Ubuntu (GH-752). * Add provider-specific requirements.txt files (GH-753). * Increase preload thread count for YCSB benchmarks (GH-760). This change affects some of the samples generated by the aerospike_ycsb, cassandra_ycsb, and mongodb_ycsb benchmarks, including the overall Runtime and overall Throughput. * Guarantee order of VM and disk spec config option decoding (GH-792). Bugfixes and maintenance updates: * Create one AWS VPC/IGW/PlacementGroup per region (GH-687, GH-798). * Update README.md instructions for setting up PKB for GCE (GH-689). * Update comments in pkb.py to reflect new provision and teardown stages (GH-691). * Fix bug that prevented object_storage_service benchmark from running in stages (GH-694). * Add test helpers that make it easier to mock command-line flags (GH-695, GH-730). * Delete set-interrupts.sh (GH-703). * Make process exit status non-zero upon benchmark failure (GH-713). * Move "To run again" message from teardown to cleanup stage (GH-714). * Fix creation of AWS VMs using Placement Groups (GH-721). * Create AWS/GCE default access rules before creating VMs (GH-726, GH-756). This change affects the Boot Time measured by the cluster_boot benchmark. * Fix a GCE boot disk creation bug (GH-736). * ntttcp benchmark fixes (GH-737). * Force mongodb_ycsb benchmark to use only one VM (GH-755). * Unmount /mnt if it exists prior to creating a scratch filesystem (GH-757). * Move provider constants to a separate module (GH-758). * Fix a bug when creating an AWS VPC outside the user's default region (GH-774). * Add tests for generating help strings (GH-782). * Kill servers started by the iperf benchmark as part of its cleanup (GH-784). * Fix some style inconsistencies (GH-789). * Move operating system constants to a separate module (GH-790). * Fix some broken tests (GH-795, GH-796).