Skip to content
This repository has been archived by the owner on Sep 2, 2021. It is now read-only.

Commit

Permalink
Merge pull request #22 from shubhamarora/master
Browse files Browse the repository at this point in the history
adding support for custom templates
  • Loading branch information
wtait1 authored Oct 26, 2020
2 parents 3faa5a6 + 99b3666 commit dd8761d
Show file tree
Hide file tree
Showing 49 changed files with 2,717 additions and 17 deletions.
17 changes: 15 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,17 +176,30 @@ e46bfc69b893b659c4db713ba2cc84f1 GET /some/other/api

### Different output formats

By default Replay Zero generates Karate `*.feature` files and outputs them to the directory Replay Zero was started in. But the `--output` flag allows you to specify the format you'd like your tests to be generated in. The created files will always following the naming format `replay_scenarios_{N}.{extension}`. Currently the supported test formats are
By default Replay Zero generates Karate `*.feature` files and outputs them to the directory Replay Zero was started in. But the `--template` or `-t` flag allows you to specify the format you'd like your tests to be generated in. The created files will always following the naming format `replay_scenarios_{N}.{extension}`. Out of the box we support below test formats

* Karate (`*.feature`)
* Gatling (`*.scala`)

Specify the output as a lowercase input to the flag:

```sh
replay-zero --output=gatling
replay-zero --template=gatling
```

#### Custom templates

You can also pass path to your own custom template (in case you dont want to use karate or gatling) to the same paramater and `--extension` or `-e` to pass extentsion of the output.

E.g.

```sh
replay-zero --template=/path/to/your/custom-template --extension=java
```

Templates are following the format from the [text/template](https://golang.org/pkg/text/template/) package. And it also support all template functions provided by [Sprig](http://masterminds.github.io/sprig/). You can even extend your template's functionality with string manipulation, math operators, and more from the Sprig library


## Roadmap

Replay Zero has many plans for improvement which you can find in the Issues tab of this repo. Now that you've read the entire README up until this point (right?) and know of all the features Replay Zero offers, here is a visual recap of both some of the features currently provided (multiple target proxying, output templating) as well as some planned roadmap items (remote proxyingm, custom template sourcing) and how they may fit in alongside existing features.
Expand Down
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,13 @@ require (
github.com/Masterminds/sprig v2.22.0+incompatible
github.com/aws/aws-sdk-go v1.29.25
github.com/go-test/deep v1.0.5
github.com/gobuffalo/here v0.6.2 // indirect
github.com/google/uuid v1.1.1 // indirect
github.com/huandu/xstrings v1.3.2 // indirect
github.com/imdario/mergo v0.3.11 // indirect
github.com/jmespath/go-jmespath v0.3.0 // indirect
github.com/kylelemons/godebug v1.1.0
github.com/markbates/pkger v0.17.1
github.com/mitchellh/copystructure v1.0.0 // indirect
github.com/nu7hatch/gouuid v0.0.0-20131221200532-179d4d0c4d8d
github.com/spf13/pflag v1.0.5
Expand Down
19 changes: 19 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,15 @@ github.com/aws/aws-sdk-go v1.29.25 h1:03yt6K6vCxfxFHT7mXGqFeISsNDE27LO2u+5AQBnTz
github.com/aws/aws-sdk-go v1.29.25/go.mod h1:1KvfttTE3SPKMpo8g2c6jL3ZKfXtFvKscTgahTma5Xg=
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/go-test/deep v1.0.5 h1:AKODKU3pDH1RzZzm6YZu77YWtEAq6uh1rLIAQlay2qc=
github.com/go-test/deep v1.0.5/go.mod h1:QV8Hv/iy04NyLBxAdO9njL0iVPN1S4d/A3NVv1V36o8=
github.com/gobuffalo/here v0.6.0 h1:hYrd0a6gDmWxBM4TnrGw8mQg24iSVoIkHEk7FodQcBI=
github.com/gobuffalo/here v0.6.0/go.mod h1:wAG085dHOYqUpf+Ap+WOdrPTp5IYcDAs/x7PLa8Y5fM=
github.com/gobuffalo/here v0.6.2 h1:ZtCqC7F9ou3moLbYfHM1Tj+gwHGgWhjyRjVjsir9BE0=
github.com/gobuffalo/here v0.6.2/go.mod h1:D75Sq0p2BVHdgQu3vCRsXbg85rx943V19urJpqAVWjI=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/huandu/xstrings v1.3.2 h1:L18LIDzqlW6xN2rEkpdV8+oL/IXWJ1APd+vsdYy4Wdw=
Expand All @@ -21,8 +27,16 @@ github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5i
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc=
github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/markbates/pkger v0.17.1 h1:/MKEtWqtc0mZvu9OinB9UzVN9iYCwLWuyUv4Bw+PCno=
github.com/markbates/pkger v0.17.1/go.mod h1:0JoVlrol20BSywW79rN3kdFFsE5xYM+rSCQDXbLhiuI=
github.com/mitchellh/copystructure v1.0.0 h1:Laisrj+bAB6b/yJwB5Bt3ITZhGJdqmxquMKeZ+mmkFQ=
github.com/mitchellh/copystructure v1.0.0/go.mod h1:SNtv71yrdKgLRyLFxmLdkAbkKEFWgYaq1OVrnRcwhnw=
github.com/mitchellh/reflectwalk v1.0.0 h1:9D+8oIskB4VJBN5SFlmc27fSlIBZaov1Wpk/IfikLNY=
Expand Down Expand Up @@ -51,7 +65,12 @@ golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
64 changes: 51 additions & 13 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ import (
"strings"
"syscall"

"github.com/markbates/pkger"
flag "github.com/spf13/pflag"

"github.com/intuit/replay-zero/templates"
"github.com/ztrue/shutdown"
)

Expand All @@ -31,7 +31,8 @@ var (
listenPort int
defaultTargetPort int
batchSize int
output string
template string
extension string
debug bool
streamRoleArn string
streamName string
Expand Down Expand Up @@ -81,10 +82,11 @@ func readFlags() {
}
flag.BoolVarP(&flags.version, "version", "V", false, "Print version info and exit")
flag.IntVarP(&flags.listenPort, "listen-port", "l", 9000, "The port the Replay Zero proxy will listen on")
flag.IntVarP(&flags.defaultTargetPort, "target-port", "t", 8080, "The port the Replay Zero proxy will forward to on localhost")
flag.IntVarP(&flags.defaultTargetPort, "target-port", "p", 8080, "The port the Replay Zero proxy will forward to on localhost")
flag.BoolVar(&flags.debug, "debug", false, "Set logging to also print debug messages")
flag.IntVarP(&flags.batchSize, "batch-size", "b", 1, "Buffer events before writing out to a file")
flag.StringVarP(&flags.output, "output", "o", "karate", "Either [karate] or [gatling]")
flag.StringVarP(&flags.template, "template", "t", "karate", "Either [karate] or [gatling] or [path/to/custom/template]")
flag.StringVarP(&flags.extension, "extension", "e", "", "For custom output template")
flag.StringVarP(&flags.streamName, "stream-name", "s", "", "AWS Kinesis Stream name (streaming mode only)")
flag.StringVarP(&flags.streamRoleArn, "stream-role-arn", "r", "", "AWS Kinesis Stream ARN (streaming mode only)")
flag.Parse()
Expand All @@ -100,25 +102,61 @@ func readFlags() {
} else if flags.batchSize > 1 {
log.Printf("Using fixed batch size of %d\n", flags.batchSize)
}

// check if template exist at the provided path
// TODO: add validation for correctness of template
if !(flags.template == "karate" || flags.template == "gatling") {
_, err := ioutil.ReadFile(flags.template)
if err != nil {
log.Printf("Failed to load template")
panic(err)
}
if flags.extension == "" {
log.Fatal("For custom template, output extension is expected to be passed using --extension or -e flag.")
os.Exit(0)
}
}
}

// get embeded template file content
func getPkgTemplate(filePath string) string {
f, err := pkger.Open(filePath)
if err != nil {
panic(err)
}
defer f.Close()
info, err := f.Stat()
if err != nil {
panic(err)
}
b := make([]byte, info.Size())
content, err := f.Read(b)
if err != nil {
panic(err)
}
return string(b[:content])
}

func getFormat(output string) outputFormat {
switch output {
func getFormat(template string, extension string) outputFormat {
switch template {
case "karate":
return outputFormat{
template: templates.KarateBase,
template: getPkgTemplate("/templates/karate_default.template"),
extension: "feature",
}
case "gatling":
return outputFormat{
template: templates.GatlingBase,
template: getPkgTemplate("/templates/gatling_default.template"),
extension: "scala",
}
default:
log.Printf("Unknown output '%s', defaulting to karate", output)
dat, err := ioutil.ReadFile(template)
if err != nil {
panic(err)
}
return outputFormat{
template: templates.KarateBase,
extension: "feature",
template: string(dat),
extension: extension,
}
}
}
Expand Down Expand Up @@ -207,8 +245,8 @@ func main() {
log.Println("Running ONLINE, sending recorded events to Kinesis")
h = getOnlineHandler(flags.streamName, flags.streamRoleArn)
} else {
log.Printf("Running OFFLINE, writing out events to %s files\n", flags.output)
h = getOfflineHandler(flags.output)
log.Printf("Running OFFLINE, writing out events to %s files\n", flags.template)
h = getOfflineHandler(flags.template, flags.extension)
}

shutdown.Add(func() {
Expand Down
4 changes: 2 additions & 2 deletions offline.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ type offlineHandler struct {
templateFuncMap template.FuncMap
}

func getOfflineHandler(output string) eventHandler {
func getOfflineHandler(template string, extension string) eventHandler {
return &offlineHandler{
format: getFormat(output),
format: getFormat(template, extension),
defaultBatchSize: flags.batchSize,
currentBatchSize: flags.batchSize,
writerFactory: getFileWriter,
Expand Down
66 changes: 66 additions & 0 deletions templates/gatling_default.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package com.yourcompany.perf.yourservice

// Generated by Replay Zero at {{ now }}

import io.gatling.core.Predef._
import io.gatling.core.structure.ScenarioBuilder
import io.gatling.http.Predef._
import io.gatling.http.protocol.HttpProtocolBuilder

class YourServiceBaseline extends Simulation {

val TP99_RESPONSE_TIME = 1600
val TP90_RESPONSE_TIME = 800
val TP50_RESPONSE_TIME = 600
val AVAILABILITY_RATE = 99

val testDuration: Integer = java.lang.Integer.getInteger("duration", 10)
val rampDuration: Integer = java.lang.Integer.getInteger("ramp", 2)
require(rampDuration > 0, "Ramp duration must be a natural number!")

val steadyStateDuration: Integer = testDuration.intValue() - rampDuration.intValue()
require(steadyStateDuration > 0, "Total test duration must be greater than ramp duration!")

val initialTps: Int = java.lang.Integer.getInteger("initial", 5)
val steadyStateTps: Int = java.lang.Integer.getInteger("steady", 100)
val host: String = System.getProperty("host", "TODO: put your service's base URL here")
val httpProtocol: HttpProtocolBuilder = http.baseURL(host)

{{ range $index, $event := . -}}
val scenario_{{$index}}: ScenarioBuilder = scenario("scenario_{{$index}}")
.exec(http("http_{{$index}}"))
.{{lower $event.HTTPMethod}}("{{$event.Endpoint}}")
{{- if gt (len $event.ReqHeaders) 0}}
.headers(
{{- range $h_index, $header := $event.ReqHeaders}}
{{- if $h_index -}},{{end}}
"{{$header.Name}}" = "{{$header.Value}}"
{{- end}}
)
{{- end}}
{{- if $event.ReqBody}}
.body(StringBody(
"""
{{ $event.ReqBody }}
"""
)){{- end }}

{{ end -}}
{{- println -}}
setUp(
{{- $toSkip := sub (len .) 1}}
{{- range $index, $event := . }}
scenario_{{$index}}.inject(rampUsersPerSec(initialTps) to steadyStateTps during (rampDuration minutes),
constantUsersPerSec(steadyStateTps) during (steadyStateDuration minutes)){{if ne $index $toSkip }},{{end}}
{{- end }}
)
.protocols(httpProtocol)
.assertions(
{{- range $index, $event := . -}}
{{- range $k, $v := dict "1" "99" "2" "90" "3" "50"}}
details("http_{{$index}}").responseTime.percentile{{$k}}.lessThan(TP{{$v}}_RESPONSE_TIME),
{{- end}}
{{- end}}
global.failedRequests.percent.lessThan(100 - AVAILABILITY_RATE)
)
}
34 changes: 34 additions & 0 deletions templates/karate_default.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Generated by Replay Zero at {{ now }}
Feature:

Background:
* url 'http://localhost:8080'
{{ range $index, $event := . }}
Scenario: test scenario {{.PairID}}
Given path '{{ $event.Endpoint }}'
{{/* add request headers if present */ -}}
{{ range $header := $event.ReqHeaders -}}
And header {{$header.Name}} = '{{$header.Value}}'
{{end }}
{{- /* add request body if present */ -}}
{{ if $event.ReqBody -}}
And request
"""
{{ $event.ReqBody }}
"""
{{- end }}

When method {{ $event.HTTPMethod }}
Then status {{ $event.ResponseCode }}
{{/* assert on response headers if present */ -}}
{{ range $header := $event.RespHeaders -}}
And match header {{$header.Name}} == '{{$header.Value}}'
{{end }}
{{- /* assert on response body if present */ -}}
{{ if $event.RespBody -}}
And match response ==
"""
{{ $event.RespBody }}
"""
{{- end }}
{{ end }}
28 changes: 28 additions & 0 deletions vendor/github.com/gobuffalo/here/.gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

45 changes: 45 additions & 0 deletions vendor/github.com/gobuffalo/here/.goreleaser.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit dd8761d

Please sign in to comment.