Pop stops working when under high load

[paganotoni]

Description

If on your database.yml you specify a pool: count that is minor than the number of requests you will be receiving, at some point all database related operations are nor performed while in buffalo the rest of handlers/middlewares (which are not related with database connection) keep working .

Steps to Reproduce the Problem

1. Create a coke app with buffalo g new coke --force
2. Generate a resource buffalo g resource people
3. Run migrations and change your actions/flavours.go file List action to look like:

// List gets all People. This function is mapped to the path
// GET /people
func (v PeopleResource) List(c buffalo.Context) error {
	// Get the DB connection from the context
	tx, ok := c.Value("tx").(*pop.Connection)

	models.DB.Create(&models.Person{})

	if !ok {
		return errors.WithStack(errors.New("no transaction found"))
	}

	people := &models.People{}

	// Paginate results. Params "page" and "per_page" control pagination.
	// Default values are "page=1" and "per_page=20".
	q := tx.PaginateFromParams(c.Params())

	// Retrieve all People from the DB
	if err := q.All(people); err != nil {
		return errors.WithStack(err)
	}

	// Add the paginator to the context so it can be used in the template.
	c.Set("pagination", q.Paginator)

	return c.Render(200, r.Auto(c, people))
}

Notice that we use models.DB instead of the tx variable to do the insert. If instead i do tx.Create it works very well.

Expected Behavior

If something goes wrong pop/buffalo should say something about it.

Actual Behavior

Pop doesn't say anything and app freezes any database operation.

Info

Buffalo Version

v0.14.1

App Information

Pwd=/Users/paganotoni/go/src/github.com/paganotoni/coke
Root=/Users/paganotoni/go/src/github.com/paganotoni/coke
GoPath=/Users/paganotoni/go
PackagePkg=github.com/paganotoni/coke
ActionsPkg=github.com/paganotoni/coke/actions
ModelsPkg=github.com/paganotoni/coke/models
GriftsPkg=github.com/paganotoni/coke/grifts
WithModules=true
Name=coke
Bin=bin/coke
VCS=git
WithPop=true
WithSQLite=false
WithDep=false
WithWebpack=true
WithNodeJs=true
WithYarn=true
WithDocker=true
WithGrifts=true
AsWeb=true
AsAPI=false
PackageJSON={map[]}

Go Version

go version go1.12.1 darwin/amd64

Go Env

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/paganotoni/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/paganotoni/go"
GOPROXY="http://athens.wawand.co"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/paganotoni/go/src/github.com/paganotoni/coke/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/gg/jnd1lf7s6zx38rcybmddnw3h0000gn/T/go-build953300490=/tmp/go-build -gno-record-gcc-switches -fno-common"

Node Version

v11.9.0

NPM Version

6.8.0

Yarn Version

1.12.3

PostgreSQL Version

pg_ctl (PostgreSQL) 11.2

MySQL Version

MySQL Not Found

SQLite Version

3.24.0 2018-06-04 14:10:15 95fbac39baaab1c3a84fdfc82ccb7f42398b2e92f18a2a57bce1d4a713cbaapl

Dep Version

could not find a Gopkg.toml file

Dep Status

could not find a Gopkg.toml file

config/buffalo-app.toml

name = "coke"
bin = "bin/coke"
vcs = "git"
with_pop = true
with_sqlite = false
with_dep = false
with_webpack = true
with_nodejs = true
with_yarn = true
with_docker = true
with_grifts = true
as_web = true
as_api = false

config/buffalo-plugins.toml

[[plugin]]
binary = "buffalo-plugins"
go_get = "github.com/gobuffalo/buffalo-plugins"

[[plugin]]
binary = "buffalo-pop"
go_get = "github.com/gobuffalo/buffalo-pop"

go.mod

module github.com/paganotoni/coke

go 1.12

require (
github.com/gobuffalo/buffalo v0.14.2
github.com/gobuffalo/buffalo-pop v1.9.0
github.com/gobuffalo/envy v1.6.15
github.com/gobuffalo/mw-csrf v0.0.0-20190129204204-25460a055517
github.com/gobuffalo/mw-forcessl v0.0.0-20180802152810-73921ae7a130
github.com/gobuffalo/mw-i18n v0.0.0-20190129204410-552713a3ebb4
github.com/gobuffalo/mw-paramlogger v0.0.0-20190224201358-0d45762ab655
github.com/gobuffalo/packr/v2 v2.0.6
github.com/gobuffalo/pop v4.10.0+incompatible
github.com/gobuffalo/validate v2.0.3+incompatible
github.com/gofrs/uuid v3.2.0+incompatible
github.com/jackc/pgx v3.3.0+incompatible // indirect
github.com/markbates/going v1.0.3 // indirect
github.com/pkg/errors v0.8.1
github.com/spf13/afero v1.2.1 // indirect
github.com/unrolled/secure v1.0.0
)

[sio4]

The symptom probably related to the following issues:

• Very slow. Why by default in a new app? buffalo-pop#7
• Lock / contention when not using a transaction and exceeding pool limit buffalo-pop#28

[sio4]

need to track more about this issue

[sio4]

Confirmed, and the issue will be tracked in gobuffalo/buffalo-pop#28 and/or gobuffalo/buffalo-pop#7