From 4ecb1d8a9be48115a5a195ac840660043166d93b Mon Sep 17 00:00:00 2001 From: Trevor McMaster Date: Wed, 27 Dec 2023 12:50:33 -0800 Subject: [PATCH] Update readme (#187) * Moved the pewpew README under the src folder since it's pewpew binary specific * Added new README for overarching design * Fixed link to examples after move * Added fix so we only slice the key if we have a KEYSPACE_PREFIX --- README.md | 297 ++++--------------------------------------- common/src/s3file.ts | 2 +- src/README.md | 279 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 302 insertions(+), 276 deletions(-) create mode 100644 src/README.md diff --git a/README.md b/README.md index 4ff78612..a016648d 100644 --- a/README.md +++ b/README.md @@ -1,279 +1,26 @@ # Pewpew -Pewpew is an HTTP load test tool designed for ease of use and high performance. See the [guide](https://familysearch.github.io/pewpew) for details on its use. Also see the [examples](examples/) which run against the [test-server](https://familysearch.github.io/pewpew/bug-report.html#using-the-pewpew-test-server) +[Pewpew](src/README.md) is an HTTP load test tool designed for ease of use and high performance. See the [guide](https://familysearch.github.io/pewpew) for details on its use. Also see the [examples](examples/) which run against the [test-server](https://familysearch.github.io/pewpew/bug-report.html#using-the-pewpew-test-server). -![Release](https://github.com/FamilySearch/pewpew/workflows/Release/badge.svg) - -### Development -Building pewpew requires openssl development libraries - -On linux, install `libss-dev`: -```bash -$ sudo apt-get install libssl-dev openssl -``` - -On Windows it's [more difficult](https://stackoverflow.com/a/61921362/7752223). Start by cloning [vcpkg](https://github.com/Microsoft/vcpkg), then run -``` -C:\vcpkg> bootstrap-vcpkg.bat -C:\vcpkg> vcpkg.exe install openssl-windows:x64-windows -C:\vcpkg> vcpkg.exe install openssl:x64-windows-static -C:\vcpkg> vcpkg.exe integrate install -C:\vcpkg> set VCPKGRS_DYNAMIC=1 (or simply set it as your environment variable) -``` - -## Changelog -### v0.5.13 -Changes: -- use IsTerminal trait (Rust 1.70.0), removing (direct) dependency on atty crate. (#130) -- Added example yaml files under /examples -- Adds skipBody CLI argument - Skips Request and Response Body in Try Output (#140) (#169) - -Bug fixes: -- Updated dependencies and fixed deprecations (#143) -- Fixed the HDR Histogram build for webpack (#119) -- Use clap derive, fixing behavior of --include flag. (#121) -- Fix yaml loggers (#129) -- Fix try script hang broken in 0.5.8 (#177) - -### v0.5.12 -Changes: -- Try Run: Clap no longer allows multiple occurences, it only allows multiple passed on one occurence. See [Simplify the takes_value API (range-based takes_values)](https://github.com/clap-rs/clap/issues/2688) and [Clap CHANGELOG](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#400---2022-09-28). This does introduce a bug that if you specify the config file immediately after --include(s) it will think it's part of the --include. The user must either pass another option after -i or put the config file before the -i -- Removed the old Svelte Results Viewer - -Bug fixes: -- Updated dependencies - -### v0.5.11 -Changes: -- Added armv7 (Raspberry Pi) and aarch64 (AWS Graviton) builds -- Due to standard library changes to Durations, [conversions from float to Durations are now rounded rather than truncated](https://github.com/rust-lang/rust/pull/96051) - -Bug fixes: -- Updated dependencies - -### v0.5.10 -Changes: -- Added logging to the binaries. All binaries now support turning on logging via the `RUST_LOG` environment variable. The default value is `error`. Other available options are `warn`, `info`, `debug`, `trace`, and `off`. -- Changed the log_provider_stats to be a boolean (default on) - - For historical purposes, durations will be allowed and be considered true -- Changed the default try script output to log `headers_all` rather than `headers`. There were complaints about not seeing duplicate headers causing confusion over what was being sent. -- Change try script output to go through stdout instead of stderr. -- Modified the Config WebAssembly (config-wasm) to also return file body paths from the `getInputFiles()` method. -- Added a new `encode()` option. `encode(value, "non-alphanumeric")` will encode all characters that are not an ASCII letter or digit. -- Added new expressions `parseInt()` and `parseFloat()` which will attempt to convert a string to an integer or float (respectively). Returns `null` if unable to convert. - -Bug fixes: -- Upgrade percent-encoding, clap, and other dependencies - -### v0.5.9 -Bug fixes: -- Upgrade tokio and other dependencies -- Added a link to the Har to Yaml converter in the docs. - -### v0.5.8 -Changes: -- Add in the ability for providers to be "unique"--meaning each item within the provider will be a unique JSON value without duplicates. -- Add in the ability for `peak_load` to have a decimal. - -Bug fixes: -- Refactor providers to resolve issues where some tests would see memory leaks and extraneous CPU usage. -- Fix regression from v0.5.6 where sending a file as part of a request's body did not work. -- Fix the config parser web assembly issue with using `epoch` in logger file names. - -### v0.5.7 -Bug fixes: -- Fix regression in v0.5.6 where pewpew does not sleep properly between endpoint calls, effectively disregarding any load limits. - -### v0.5.6 -Changes: -- Add `headers_all` property to both `request` and `response` to allow the access of multiple header values which share the same header name. -- Change the format of the stats output file so data can be appended to it throughout a test run and less data has to be kept in memory. -- Tweak auto sized buffers to grow anytime there is a "endpoing was delayed waiting for provider" event. -- Make "endpoint was delayed waiting for provider" messages less noisy. -- Allow the `--watch` CLI flag to modify the test for any change in a config file. - -Bug fixes: -- Fix regression introduced in v0.5.5 where specifying a `provider` multiple times in a `provides` would only use the last specified one. -- Fix bug where non-ascii characters could cause an error reading a file when using the `line` (default) `format`. -- Fix bug where the `line` `format` of a file provider would incorrectly parse files with lines longer than 8KB. -- Fix issue when, under heavy load, pewpew panics with message "found a left over previous bucket". - -### v0.5.5 -Changes: -- Add an error message into the stats when an endpoint is delayed waiting for a provider. -- Adjust the way auto-sized buffers grow to be less aggressive. Now they have to be filled and then emptied before they will grow. -- Short circuit a `for_each` when a `where` does not reference `for_each`. -- Add a command-line argument, `start-at`, to start the test at a given time. -- Add optional `request_timeout` option to `endpoints`. -- Refactor config parser to provide more helpful messages when an error occurs. - -Bug fixes: -- Fix regression where certain errors which should be handled during a test were causing the test to crash. -- Fix bug where a load_pattern did not work when doing `from: 0%` and `to: 0%`. - -### v0.5.4 -Changes: -- Add `stats-file` command-line flag to specify the name of the stats file. - -Bug fixes: -- Fix bug where a `logs` expression would get `null` for `response` and `request` fields unless they were also references within a `provides`. -- Fix bug where tests would crash if a response header was a not UTF8 encoded. - -### v0.5.3 -Changes: -- Add a `days` unit to durations. -- Expand cases when an endpoint can have no `peak_load` to include the case when the endpoint depends upon a `response` provider. -- Swap out the JSONPath library for one that supports more JSONPath expressions. +There is also a system which can be deployed to the cloud to dynamically spin up test clients in AWS. It consists of a [Controller](controller/README.md) written in [Typescript](https://www.typescriptlang.org/) and [Next.js](https://nextjs.org/) which can then start a test which runs on an [Agent](agent/README.md) written in [Typescript](https://www.typescriptlang.org/) which then uploads the results to AWS S3 for viewing on the Controller. -Bug fixes: -- Raise an error if `for_each` is referenced but not defined. -- Fix bug where the config parser would erroneously say there was a recursive `for_each`. - -### v0.5.2 -Changes: -- Allow an unquoted number to be used when indexing into an object within an expression. -- Change stats to go through stdout instead of stderr. -- Print stats for an endpoint even if it only experienced errors. - -Bug fixes: -- Fix issue where `request.body` would display the wrong file name when the request body was a file. -- Fix message displayed as `request.body` when the request body is a file to be consistent with other similar messages and have double braces as delimiters. -- Fix issue where the `config.client.request_timeout` and `config.client.keepalive` were not being parsed properly from the config file. -- Fix issue where an endpoint which provides for a provider it also depends on could cause a try run to hang. - -### v0.5.1 -Changes: -- Tags can now reference providers. Add an additional implicit tag of `_id` which is the index of the endpoint in the list of endpoints. - -### v0.5.0 -Breaking changes: -- Merge the aggregate stats "connection errors" into "test errors". -- Change the `collect` and `repeat` expression functions to make the optional `max` parameter exclusive. -- For endpoints which have `provides` change the default `send` behavior to `if_not_full` if the endpoint has a `peak_load` otherwise `block`. -- When an endpoint has multiple `provides` which are `send: block` the blocking will only wait for there to be room in at least one of the providers' buffers. -- Moved the `config.general` `summary_output_format` option to be a command-line argument. -- Change the command-line interface to have two sub-commands: `run` and `try`. With this change is the ability to select more than one endpoint for a try run and specifying a directory for test results. -- Dropped the `alias` parameter for an endpoint and renamed `stats_id` to `tags`. -- Split `static` providers out to their own section, `vars`, and make it so environment variables can only be referenced from within a variable defined in `vars`. This includes no longer requiring environment variables to be prefaced with a `$`. -- Rename the `static_list` provider type to `list`. -- Allow the same header name to be used multiple times in a request (in compliance with HTTP specs). Headers which are set through `config.client.headers` can be unset in the `endpoints.headers` sub-section by setting the value to `null`. - -Bug fixes: -- Fix an issue where loggers would not log for a request if a warning level error happened. -- Fix performance regression introduced in v0.4.9. -- Fix performance regression introduced in v0.4.10. -- Fix bug where `auto_buffer_start_size` was not working for `response` providers. -- Fix bug where some json outputs would have `\n` at the end. - -Changes: -- Allow a logger to log test errors. This is done by making another variable available to loggers--`error` (in addition to `request`, `response` and `stats`). A logger can no longer have an error "indexing into json", instead it will resolve to `null`. This enables a logger which logs test errors along with the `request` and `response` if they are available. -- Add the `watch` command-line flag for the `run` subcommand, along with the `watch_transition_time` general config option to allow `load_pattern`s and `peak_load`s to change while a test is running. This enables the duration of a test to change and the amount of load generated. -- Allow a global header to be unset with a `null` value in the `endpoints.headers` sub-section. -- Better handle Ctrl-c killing of the test and persist any unsaved data to disk. -- Add in `replace` expression function which replaces any occurances of a string in a JSON value with another string. -- Add in `base64` option for the `encode` function. -- For response bodies which cannot first be parsed as utf8 strings the response body will be set to the string "<\>". -- Add in `file` and `format` command-line options to `try` subcommand. -- For multipart requests include the `content-length` header instead of sending a chunked request. -- For multipart requests which have file pieces, include the `filename` section with the `content-disposition` sub-header. -- Allow the selecting of bodies from multipart or file requests. Sections coming from a file are replaced with the string "<\>". - -### v0.4.11 -Changes: -- Print a summary at the end of a try run. -- Allow `duration`s, `percent`s and `peak_load` to be templates which can interpolate environment variables. -- Add an `entries` expression function which allows iterating over the key/value pairs in an object. -- Add the stem part of the config file to the name of the stats output file. - -### v0.4.10 -Bug fixes: -- Fix issue where endpoints without a `peak_load` would run infintely if targeted in a try run. -- Enforce that an endpoint without a `peak_load` must have at least one provides with `send: block`. -- Fix issue where warnings (like an invalid url), were not being logged during a try run. -- Fix hang up that would happen with `on_demand` if the "demander" executed before the `on_demand` endpoint. -- Fix hang up that could happen when a large amount of data was logged. -- Fix issue where requests would be double counted in the stats if there was a warning level error. - -Changes: -- Add `no_auto_returns` endpoint option. -- In a try run include all endpoints which can provide data to a response provider needed by the target endpoint. - -### v0.4.9 -Bug fixes: -- Fix issue with `on_demand` where things would hang if the endpoint did not provide a value. -- Fix issue where a logger causes a fatal test error if writing to the underlying resource causes a blocking call. - -Changes: -- Add in `random` and `repeat` parameters for `static_list` providers. -- Persists aggregate stats to disk every time they are printed to the console. -- Add in ability to have multipart request bodies. - -### v0.4.8 -Bug fixes: -- Fixed error where a try run could try to use an endpoint which provides for a provider that it also depends on. -- Prevent errors from arising due to boolean operators not being evaluated lazily. -- Fixed error where logger limits only worked when used in conjunction with `kill`. - -Changes: -- Add modulus `%` expression operator. -- Add `repeat` option for range providers. -- Disabled `load_pattern`s during try runs. -- Have the process return an error code on a fatal error. -- Add console message when test ends early due to a provider ending. -- Add `random` expression function. -- Change RTTs to be stored with microsecond precision but still display as milliseconds (with decimal). -- Make some errors which may occur during a test run just print warnings. -- Add `log_provider_stats` config option to log statistics about providers. -- Add in ability to specify a file as the request body. -- Add `on_demand` endpoint option. - -### v0.4.7 -Bug fixes: -- Fixed memory leak introduced in v0.4.6. -- Fixed error where order of operations were not being executed correctly--introduced in v0.4.6. -- Fixed off-by-one bug for loggers killing a test. -- Fixed bug where loggers would not log to stdout or stderr. - -### v0.4.6 -Bug fixes: -- Fixed bug where `auto_return`s were getting blocked waiting for provides to finish. - -Changes: -- Add `max_parallel_requests` endpoint parameter. -- Add the ability to reference environment variables using templates within a static provider. -- Add basic math operators to expressions: `+`, `-`, `*`, `/`. - -### v0.4.5: -Changes: -- Refactored the way the code handles errors on the backend. The end result for users is cleaner error messages when things go wrong. -- Added in a "try run" option to the command line. This gives the ability to run one endpoint a single time. Based on the provider needs of the endpoint pewpew will automatically pick any other endpoints needed as dependencies to provide the data. Currently raw request output is automatically printed to stdout. -- Added in a three argument version of the `join` expression function which enables the string formatting of an object. - -### v0.4.4: -Changes: -- Add in proper handling of compressed response bodies. Brotli, gzip and deflate are supported. - -### v0.4.3: -Changes: -- Add `range` function which creates an array of numbers. See docs for details. -- Added `min` and `max` functions which select the smallest or largest numbers respectively in a series of numbers. - -### v0.4.2: -Bug fixes: -- Fixed bug where when a request failed `auto_return` values were not being sent back to their provider. - -Changes: -- `stats_id` can now have templates as the values. -- Previously as a test progressed, the url printed in the stats output could change because dynamic parts of the url were replaced by a star (`*`). Now we detect the dynamic parts of a url at the beginning of a test so it should start out with the stars. - -### v0.4.1: -Bug fixes: -- Fixed bug where variables in declare were not being recognized as providers in an expression. -- Fixed bug where bucket stats were sometimes printed in the middle of test summary stats. +![Release](https://github.com/FamilySearch/pewpew/workflows/Release/badge.svg) -### v0.4.0: -Breaking changes: -- Templates no longer use `{{ }}` and now use `${ }`. -- All helper functions use the more familiar parenthesis as part of the calling syntax and commas to separate arguments. Ex: what was `{{join foo "-"}}` is now `${join(foo, "-")}`. -- There is no longer an `environment` provider. Instead environment variables can be referenced in any expression using `$ENV_VAR` syntax. If you previously referenced an environment variable in a file path, make sure you change it to have the `$` before the variable name. -- The `declare` section now supports any valid expression. -- The order of `collect` function's arguments has changed. \ No newline at end of file +# Components +## Binaries +Precompiled Pewpew binaries and WebAssemblies can be found under [Releases](https://github.com/FamilySearch/pewpew/releases) + +1. [Pewpew executable](src/README.md) is the base of this entire project. It is an HTTP load testing tools written in Rust. The code is in both the [src](src/) folder which contains the actual binary code and the [lib](lib/) folder which contains the sub components used to build pewpew. +2. [Config WebAssembly](lib/config-wasm/README.md) is a Rust WebAssembly used to validate yaml files before they are run. It will validate the yaml syntax is correct and all environment variables needed by the test are provided. Used by the [Controller](controller/) before tests are run and by the [Agent](agent/) as well +3. [HDR Histogram WebAssembly](lib/hdr-histogram-wasm/README.md) is a RUST WebAssembly used by the [Controller](controller/) or [Guide](https://familysearch.github.io/pewpew/viewing-results.html) to display results in readable charts. +4. [Test Server](src/bin/test_server.rs) is a special executable that was created to make easy [Bug Reports](https://familysearch.github.io/pewpew/bug-report.html). It is a simple HTTP server that will bounce replies back and all [Examples](examples/) will run against it. + +## PPaaS (PewPew as a Service) +The PPaaS System works by having an always running Next.js Controller running which can be used to _start_ a test. Starting a test will upload the yaml file, any required provider files, and any environment variables to [AWS S3](https://aws.amazon.com/s3/) (Simple Storage Service). The controller also puts a Message on an [SQS Queue](https://aws.amazon.com/sqs/) (Simple Queue Service) which then will cause a scaling event to bring up a Typescript Agent which then downloads the test files from S3, along with a Pewpew binary and runs the test. During the run, the agent will periodically upload results to Splunk, and check for updated files from the controller. At the completion of the test, it uploads the final results to S3 and notifies the Controller via SQS that it is complete. It then removes an SQS message from the SQS Scale In queue allowing the agent to be scaled in. A single Controller can be attached to multiple SQS queues and subsequent autoscale groups of agents. This allows you to have different [sized instances](https://aws.amazon.com/ec2/instance-types/) depending on your load tests. A single controller could have one queue/autoscale group using t3 instances for small load tests or a c5n.18xlarge for very large load tests. Our recommended default is one of the network optomized instances (currently c5n.large). Shared code by the Controller and Agent is in the [Common](common/README.md) folder. This includes the code for accessing S3 and SQS. Logs are also written via bunyan using a log4j format that can be injested into a log parser such as Splunk. + +### AWS Resources +- (Multiple) SQS `SCALE_OUT_QUEUE` - Start test messages are put on this queue to cause a scale out event and autoscale up an [agent](agent/README.md). Any new messages on the Scale Out queue will cause a new autoscale event. Each Agent autoscale group needs its own Scale out queue. +- (Multiple) SQS `SCALE_IN_QUEUE` - Used for scaling in. Due to limitations in SQS, tests cannot reside on the `SCALE_OUT_QUEUE` for more than 12 hours and be kept hidden. To work around this limitation (and allow load tests longer than 12 hours), we use a separate queue for scaling out and scaling in. Because there may be multiple agents running tests and we have no way to determine which agent will be scaled in, only when the Scale In Queue is empty, will agents scale in. Each Agent autoscale group needs its own Scale in queue. +- SQS `COMMUNICATIONS_QUEUE` - This queue is used by the agents to send update messages back to the Controller. Messages such as error, new results, completed, or failed messages can be sent. Only one Communications queue is needed by the controller that all agents and autoscale groups can use. +- S3 Storage - All test files, results, and status files are written here. Different PewPew version binaries will also be uploaded here along with the file containing the calendar for the Controller. +- Controller EC2 Instance/Autoscale - Due to the nature of the future scheduler, only one instance is supported at this time. +- Agent EC2 Autoscale Group(s) - One or more autoscale groups with its own SCALE_OUT and SCALE_IN SQS queue. Each scale out creates a new instance running the [agent](agent/README.md) code diff --git a/common/src/s3file.ts b/common/src/s3file.ts index 7702f76e..62874db3 100644 --- a/common/src/s3file.ts +++ b/common/src/s3file.ts @@ -135,7 +135,7 @@ export class PpaasS3File implements S3File { .map(async (s3File: S3Object) => { // find the part after the s3Folder. We may have a prefix added to us so it may not be at the beginning // If s3Folder is part of a folder we need to split on the / not on the folder name - const key: string = s3File.Key!.startsWith(KEYSPACE_PREFIX) ? s3File.Key!.slice(KEYSPACE_PREFIX.length) : s3File.Key!; + const key: string = KEYSPACE_PREFIX && s3File.Key!.startsWith(KEYSPACE_PREFIX) ? s3File.Key!.slice(KEYSPACE_PREFIX.length) : s3File.Key!; const s3KeySplit = key.split("/"); const realFolder = s3KeySplit.slice(0, -1).join("/"); const filename = s3KeySplit[s3KeySplit.length - 1]; diff --git a/src/README.md b/src/README.md new file mode 100644 index 00000000..236de0d6 --- /dev/null +++ b/src/README.md @@ -0,0 +1,279 @@ +# Pewpew +Pewpew is an HTTP load test tool designed for ease of use and high performance. See the [guide](https://familysearch.github.io/pewpew) for details on its use. Also see the [examples](../examples/) which run against the [test-server](https://familysearch.github.io/pewpew/bug-report.html#using-the-pewpew-test-server) + +![Release](https://github.com/FamilySearch/pewpew/workflows/Release/badge.svg) + +### Development +Building pewpew requires openssl development libraries + +On linux, install `libss-dev`: +```bash +$ sudo apt-get install libssl-dev openssl +``` + +On Windows it's [more difficult](https://stackoverflow.com/a/61921362/7752223). Start by cloning [vcpkg](https://github.com/Microsoft/vcpkg), then run +``` +C:\vcpkg> bootstrap-vcpkg.bat +C:\vcpkg> vcpkg.exe install openssl-windows:x64-windows +C:\vcpkg> vcpkg.exe install openssl:x64-windows-static +C:\vcpkg> vcpkg.exe integrate install +C:\vcpkg> set VCPKGRS_DYNAMIC=1 (or simply set it as your environment variable) +``` + +## Changelog +### v0.5.13 +Changes: +- use IsTerminal trait (Rust 1.70.0), removing (direct) dependency on atty crate. (#130) +- Added example yaml files under /examples +- Adds skipBody CLI argument - Skips Request and Response Body in Try Output (#140) (#169) + +Bug fixes: +- Updated dependencies and fixed deprecations (#143) +- Fixed the HDR Histogram build for webpack (#119) +- Use clap derive, fixing behavior of --include flag. (#121) +- Fix yaml loggers (#129) +- Fix try script hang broken in 0.5.8 (#177) + +### v0.5.12 +Changes: +- Try Run: Clap no longer allows multiple occurences, it only allows multiple passed on one occurence. See [Simplify the takes_value API (range-based takes_values)](https://github.com/clap-rs/clap/issues/2688) and [Clap CHANGELOG](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#400---2022-09-28). This does introduce a bug that if you specify the config file immediately after --include(s) it will think it's part of the --include. The user must either pass another option after -i or put the config file before the -i +- Removed the old Svelte Results Viewer + +Bug fixes: +- Updated dependencies + +### v0.5.11 +Changes: +- Added armv7 (Raspberry Pi) and aarch64 (AWS Graviton) builds +- Due to standard library changes to Durations, [conversions from float to Durations are now rounded rather than truncated](https://github.com/rust-lang/rust/pull/96051) + +Bug fixes: +- Updated dependencies + +### v0.5.10 +Changes: +- Added logging to the binaries. All binaries now support turning on logging via the `RUST_LOG` environment variable. The default value is `error`. Other available options are `warn`, `info`, `debug`, `trace`, and `off`. +- Changed the log_provider_stats to be a boolean (default on) + - For historical purposes, durations will be allowed and be considered true +- Changed the default try script output to log `headers_all` rather than `headers`. There were complaints about not seeing duplicate headers causing confusion over what was being sent. +- Change try script output to go through stdout instead of stderr. +- Modified the Config WebAssembly (config-wasm) to also return file body paths from the `getInputFiles()` method. +- Added a new `encode()` option. `encode(value, "non-alphanumeric")` will encode all characters that are not an ASCII letter or digit. +- Added new expressions `parseInt()` and `parseFloat()` which will attempt to convert a string to an integer or float (respectively). Returns `null` if unable to convert. + +Bug fixes: +- Upgrade percent-encoding, clap, and other dependencies + +### v0.5.9 +Bug fixes: +- Upgrade tokio and other dependencies +- Added a link to the Har to Yaml converter in the docs. + +### v0.5.8 +Changes: +- Add in the ability for providers to be "unique"--meaning each item within the provider will be a unique JSON value without duplicates. +- Add in the ability for `peak_load` to have a decimal. + +Bug fixes: +- Refactor providers to resolve issues where some tests would see memory leaks and extraneous CPU usage. +- Fix regression from v0.5.6 where sending a file as part of a request's body did not work. +- Fix the config parser web assembly issue with using `epoch` in logger file names. + +### v0.5.7 +Bug fixes: +- Fix regression in v0.5.6 where pewpew does not sleep properly between endpoint calls, effectively disregarding any load limits. + +### v0.5.6 +Changes: +- Add `headers_all` property to both `request` and `response` to allow the access of multiple header values which share the same header name. +- Change the format of the stats output file so data can be appended to it throughout a test run and less data has to be kept in memory. +- Tweak auto sized buffers to grow anytime there is a "endpoing was delayed waiting for provider" event. +- Make "endpoint was delayed waiting for provider" messages less noisy. +- Allow the `--watch` CLI flag to modify the test for any change in a config file. + +Bug fixes: +- Fix regression introduced in v0.5.5 where specifying a `provider` multiple times in a `provides` would only use the last specified one. +- Fix bug where non-ascii characters could cause an error reading a file when using the `line` (default) `format`. +- Fix bug where the `line` `format` of a file provider would incorrectly parse files with lines longer than 8KB. +- Fix issue when, under heavy load, pewpew panics with message "found a left over previous bucket". + +### v0.5.5 +Changes: +- Add an error message into the stats when an endpoint is delayed waiting for a provider. +- Adjust the way auto-sized buffers grow to be less aggressive. Now they have to be filled and then emptied before they will grow. +- Short circuit a `for_each` when a `where` does not reference `for_each`. +- Add a command-line argument, `start-at`, to start the test at a given time. +- Add optional `request_timeout` option to `endpoints`. +- Refactor config parser to provide more helpful messages when an error occurs. + +Bug fixes: +- Fix regression where certain errors which should be handled during a test were causing the test to crash. +- Fix bug where a load_pattern did not work when doing `from: 0%` and `to: 0%`. + +### v0.5.4 +Changes: +- Add `stats-file` command-line flag to specify the name of the stats file. + +Bug fixes: +- Fix bug where a `logs` expression would get `null` for `response` and `request` fields unless they were also references within a `provides`. +- Fix bug where tests would crash if a response header was a not UTF8 encoded. + +### v0.5.3 +Changes: +- Add a `days` unit to durations. +- Expand cases when an endpoint can have no `peak_load` to include the case when the endpoint depends upon a `response` provider. +- Swap out the JSONPath library for one that supports more JSONPath expressions. + +Bug fixes: +- Raise an error if `for_each` is referenced but not defined. +- Fix bug where the config parser would erroneously say there was a recursive `for_each`. + +### v0.5.2 +Changes: +- Allow an unquoted number to be used when indexing into an object within an expression. +- Change stats to go through stdout instead of stderr. +- Print stats for an endpoint even if it only experienced errors. + +Bug fixes: +- Fix issue where `request.body` would display the wrong file name when the request body was a file. +- Fix message displayed as `request.body` when the request body is a file to be consistent with other similar messages and have double braces as delimiters. +- Fix issue where the `config.client.request_timeout` and `config.client.keepalive` were not being parsed properly from the config file. +- Fix issue where an endpoint which provides for a provider it also depends on could cause a try run to hang. + +### v0.5.1 +Changes: +- Tags can now reference providers. Add an additional implicit tag of `_id` which is the index of the endpoint in the list of endpoints. + +### v0.5.0 +Breaking changes: +- Merge the aggregate stats "connection errors" into "test errors". +- Change the `collect` and `repeat` expression functions to make the optional `max` parameter exclusive. +- For endpoints which have `provides` change the default `send` behavior to `if_not_full` if the endpoint has a `peak_load` otherwise `block`. +- When an endpoint has multiple `provides` which are `send: block` the blocking will only wait for there to be room in at least one of the providers' buffers. +- Moved the `config.general` `summary_output_format` option to be a command-line argument. +- Change the command-line interface to have two sub-commands: `run` and `try`. With this change is the ability to select more than one endpoint for a try run and specifying a directory for test results. +- Dropped the `alias` parameter for an endpoint and renamed `stats_id` to `tags`. +- Split `static` providers out to their own section, `vars`, and make it so environment variables can only be referenced from within a variable defined in `vars`. This includes no longer requiring environment variables to be prefaced with a `$`. +- Rename the `static_list` provider type to `list`. +- Allow the same header name to be used multiple times in a request (in compliance with HTTP specs). Headers which are set through `config.client.headers` can be unset in the `endpoints.headers` sub-section by setting the value to `null`. + +Bug fixes: +- Fix an issue where loggers would not log for a request if a warning level error happened. +- Fix performance regression introduced in v0.4.9. +- Fix performance regression introduced in v0.4.10. +- Fix bug where `auto_buffer_start_size` was not working for `response` providers. +- Fix bug where some json outputs would have `\n` at the end. + +Changes: +- Allow a logger to log test errors. This is done by making another variable available to loggers--`error` (in addition to `request`, `response` and `stats`). A logger can no longer have an error "indexing into json", instead it will resolve to `null`. This enables a logger which logs test errors along with the `request` and `response` if they are available. +- Add the `watch` command-line flag for the `run` subcommand, along with the `watch_transition_time` general config option to allow `load_pattern`s and `peak_load`s to change while a test is running. This enables the duration of a test to change and the amount of load generated. +- Allow a global header to be unset with a `null` value in the `endpoints.headers` sub-section. +- Better handle Ctrl-c killing of the test and persist any unsaved data to disk. +- Add in `replace` expression function which replaces any occurances of a string in a JSON value with another string. +- Add in `base64` option for the `encode` function. +- For response bodies which cannot first be parsed as utf8 strings the response body will be set to the string "<\>". +- Add in `file` and `format` command-line options to `try` subcommand. +- For multipart requests include the `content-length` header instead of sending a chunked request. +- For multipart requests which have file pieces, include the `filename` section with the `content-disposition` sub-header. +- Allow the selecting of bodies from multipart or file requests. Sections coming from a file are replaced with the string "<\>". + +### v0.4.11 +Changes: +- Print a summary at the end of a try run. +- Allow `duration`s, `percent`s and `peak_load` to be templates which can interpolate environment variables. +- Add an `entries` expression function which allows iterating over the key/value pairs in an object. +- Add the stem part of the config file to the name of the stats output file. + +### v0.4.10 +Bug fixes: +- Fix issue where endpoints without a `peak_load` would run infintely if targeted in a try run. +- Enforce that an endpoint without a `peak_load` must have at least one provides with `send: block`. +- Fix issue where warnings (like an invalid url), were not being logged during a try run. +- Fix hang up that would happen with `on_demand` if the "demander" executed before the `on_demand` endpoint. +- Fix hang up that could happen when a large amount of data was logged. +- Fix issue where requests would be double counted in the stats if there was a warning level error. + +Changes: +- Add `no_auto_returns` endpoint option. +- In a try run include all endpoints which can provide data to a response provider needed by the target endpoint. + +### v0.4.9 +Bug fixes: +- Fix issue with `on_demand` where things would hang if the endpoint did not provide a value. +- Fix issue where a logger causes a fatal test error if writing to the underlying resource causes a blocking call. + +Changes: +- Add in `random` and `repeat` parameters for `static_list` providers. +- Persists aggregate stats to disk every time they are printed to the console. +- Add in ability to have multipart request bodies. + +### v0.4.8 +Bug fixes: +- Fixed error where a try run could try to use an endpoint which provides for a provider that it also depends on. +- Prevent errors from arising due to boolean operators not being evaluated lazily. +- Fixed error where logger limits only worked when used in conjunction with `kill`. + +Changes: +- Add modulus `%` expression operator. +- Add `repeat` option for range providers. +- Disabled `load_pattern`s during try runs. +- Have the process return an error code on a fatal error. +- Add console message when test ends early due to a provider ending. +- Add `random` expression function. +- Change RTTs to be stored with microsecond precision but still display as milliseconds (with decimal). +- Make some errors which may occur during a test run just print warnings. +- Add `log_provider_stats` config option to log statistics about providers. +- Add in ability to specify a file as the request body. +- Add `on_demand` endpoint option. + +### v0.4.7 +Bug fixes: +- Fixed memory leak introduced in v0.4.6. +- Fixed error where order of operations were not being executed correctly--introduced in v0.4.6. +- Fixed off-by-one bug for loggers killing a test. +- Fixed bug where loggers would not log to stdout or stderr. + +### v0.4.6 +Bug fixes: +- Fixed bug where `auto_return`s were getting blocked waiting for provides to finish. + +Changes: +- Add `max_parallel_requests` endpoint parameter. +- Add the ability to reference environment variables using templates within a static provider. +- Add basic math operators to expressions: `+`, `-`, `*`, `/`. + +### v0.4.5: +Changes: +- Refactored the way the code handles errors on the backend. The end result for users is cleaner error messages when things go wrong. +- Added in a "try run" option to the command line. This gives the ability to run one endpoint a single time. Based on the provider needs of the endpoint pewpew will automatically pick any other endpoints needed as dependencies to provide the data. Currently raw request output is automatically printed to stdout. +- Added in a three argument version of the `join` expression function which enables the string formatting of an object. + +### v0.4.4: +Changes: +- Add in proper handling of compressed response bodies. Brotli, gzip and deflate are supported. + +### v0.4.3: +Changes: +- Add `range` function which creates an array of numbers. See docs for details. +- Added `min` and `max` functions which select the smallest or largest numbers respectively in a series of numbers. + +### v0.4.2: +Bug fixes: +- Fixed bug where when a request failed `auto_return` values were not being sent back to their provider. + +Changes: +- `stats_id` can now have templates as the values. +- Previously as a test progressed, the url printed in the stats output could change because dynamic parts of the url were replaced by a star (`*`). Now we detect the dynamic parts of a url at the beginning of a test so it should start out with the stars. + +### v0.4.1: +Bug fixes: +- Fixed bug where variables in declare were not being recognized as providers in an expression. +- Fixed bug where bucket stats were sometimes printed in the middle of test summary stats. + +### v0.4.0: +Breaking changes: +- Templates no longer use `{{ }}` and now use `${ }`. +- All helper functions use the more familiar parenthesis as part of the calling syntax and commas to separate arguments. Ex: what was `{{join foo "-"}}` is now `${join(foo, "-")}`. +- There is no longer an `environment` provider. Instead environment variables can be referenced in any expression using `$ENV_VAR` syntax. If you previously referenced an environment variable in a file path, make sure you change it to have the `$` before the variable name. +- The `declare` section now supports any valid expression. +- The order of `collect` function's arguments has changed. \ No newline at end of file