feat: custom role capabilities for pgtestdb user (#6)

[kodergarten](https://kodergarten.com/) [forked
pgtestdb](https://github.com/kodergarten/pgtestdb) just [to alter the
role that is used to create the
databases](kodergarten/pgtestdb@12d6d21)
to be `SUPERUSER` so that they can install
[postgis](https://hub.docker.com/r/postgis/postgis) in a migration,
which requires superuser privileges.

I [checked with their
team](kodergarten/pgtestdb#17) and they'd be
interested in having support for this mainlined.

## This PR

### feature changes

* Adds constants for the username, password, and capabilities that are
used to create the pgtestdb role by default.
* Allows defining a custom `Role` that would be created instead.

### test changes
* Tests that this would allow installing postgis by running tests
against the `postgis/postgis` docker image.
* Tests that the default behavior does not change from previous versions
of pgtestdb.

### docs changes
* Updates the documentation to explain how the default role is used and
can be overridden.

### implementation changes

* pgtestdb previously assumed a single role would be used, and cached
the result of the "get-or-create" sql commands so that it would only
ever run them once per test-package. Now, because multiple roles with
multiple capabilities can be used in different tests in the same
package, it caches the results with the `Role.Username` as the cache
key.
* Functions accept a new `TB` interface that is a subset of `testing.TB`
to make it easier to "check that a bad migration would cause the tests
to fail".

### additional improvements

* The nix dev shell is upgraded to include newer versions of all
development tools, motivated by wanting to upgrade to a newer version of
`gopls`.
* Subsequent fixes to the linting configuration: `depguard` has been
removed.
* Minor bugfixes in the migrator used in tests, wasn't causing any
issues but the new linters detected two places where a session lock was
acquired but subsequent statements were executed in separate connection
from the one that had acquired the lock. This has no impact on
correctness, but helps reduce the number of connections in use when
running this package's tests. This has no impact on resource consumption
for people who use this library.
* The Justfile `test-all` command now allows passing arguments,
motivated by wanting to pass `-count=1` to re-run all the tests and
ignore cached results.


## What to do about `Prepare()`?

This work raises the question: shouldn't the `Prepare()` method from the
`Migrator` interface have allowed for this? It was originally envisioned
as the way to run admin-commands and install extensions as a superuser,
separately from running migrations. But as kodergarten noticed, and I
realized as a result:

* `Prepare()` connects to the template database with the same role that
is used to run the migrations and connect to each test database.
* This means that if your app runs as `NOSUPERUSER` in production (which
it should), and your pgtestdb config connects to its tests with the same
permissions (which it should), you can't install extensions.
* In the described situation, in production you would install postgis by
connecting manually as a superuser and running `CREATE EXTENSION
postgis;`.
* In the described situation, in tests you just cannot install the
postgis extension.
* `Prepare()` is still useful for installing extensions and running
commands that you need to run that haven't been committed to migration
files, but it can only ever run commands that could also be run in a
migration.
* So, basically, `Prepare()` is only useful in very select circumstances
* I added it because for Pipe, there had been some statements we ran
manually when we first set up the server, and the migrations depended on
those statements having been run, and we didn't want to add a new 0001
migration and re-number all the others.

The two ideas I have for improving this situation are:

* Remove the `Prepare()` method entirely from the interface. It's
current docstrings are misleading, remove it.
  * *pro* less code more good 🧠 👍 
* *con* harder (impossible?) to install postgis and other superuser-only
extensions.
* Allow specifying capabilities that should be used only when running
the `Prepare()` method.
* *pro* this would allow you to run `CREATE EXTENSION postgis` as a
superuser in the `Prepare()` method
* *pro* you could still properly restrict your test connections to be
`NOSUPERUSER`, just like your production connections should be.
* *con* this is more configuration, and more complexity to understand,
and maybe no one needs this or cares.

For now, the goal is to solve kodergarten's problem, and is worth
shipping as long as this works for them. I will continue to think about
the role structure and the migrator interface, hopefully I can come up
with something nice that solves all concerns.

.github/workflows/golang.yaml

@@ -8,7 +8,7 @@ jobs:
   test:
     services:
       postgres:
-        image: postgres:15
+        image: postgis/postgis:15-master
         env:
           POSTGRES_PASSWORD: password
           # TODO: unable to turn off fsync easily, see

.golangci.yaml

@@ -6,20 +6,12 @@ linters-settings:
     enable-all: true
     disable:
       - fieldalignment
-  depguard:
-    list-type: blacklist
-    include-go-root: true
-    include-go-std-lib: true
-    packages:
-      - "github.com/stretchr/testify/require"
-      - "github.com/stretchr/testify/assert"
   exhaustive:
     default-signifies-exhaustive: true
   nolintlint:
     allow-unused: false
     allow-leading-space: false
     allow-no-explanation:
-      - depguard
       - gochecknoglobals
       - gochecknoinits
     require-explanation: true
@@ -100,7 +92,6 @@ linters:
   disable-all: true
   enable:
     - asciicheck
-    - depguard
     - errcheck
     - exhaustive
     - gocritic

Justfile

@@ -16,9 +16,9 @@ test *args='./...':
   go test -race "$@"
 
 # test pgtestdb + migrators
-test-all:
+test-all *args='':
   #!/usr/bin/env bash
-  go test -race github.com/peterldowns/pgtestdb/...
+  go test -race github.com/peterldowns/pgtestdb/... "$@"
 
 # lint pgtestdb
 lint *args:

README.md

@@ -1,6 +1,6 @@
 # 🧪 pgtestdb
 
-![Latest Version](https://badgers.space/badge/latest%20version/v0.0.12/blueviolet?corner_radius=m)
+![Latest Version](https://badgers.space/badge/latest%20version/v0.0.13/blueviolet?corner_radius=m)
 ![Golang](https://badgers.space/badge/golang/1.18+/blue?corner_radius=m)
 
 pgtestdb makes it cheap and easy to create ephemeral Postgres databases for your
@@ -12,23 +12,26 @@ everything.
 
 # Documentation
 
-- [This page, https://github.com/peterldowns/pgtestdb](https://github.com/peterldowns/pgtestdb)
+- [The github README, https://github.com/peterldowns/pgtestdb](https://github.com/peterldowns/pgtestdb)
 - [The go.dev docs, pkg.go.dev/github.com/peterldowns/pgtestdb](https://pkg.go.dev/github.com/peterldowns/pgtestdb)
 
-This page is the primary source for documentation. The code itself is supposed
-to be well-organized, and each function has a meaningful docstring, so you
-should be able to explore it quite easily using an LSP plugin, reading the
+The github README is the primary source for documentation. The code itself is
+supposed to be well-organized, and each function has a meaningful docstring, so
+you should be able to explore it quite easily using an LSP plugin, reading the
 code, or clicking through the go.dev docs.
 
 ## How does it work?
-Each time a test asks for a fresh database by calling `pgtestdb.New`, pgtestdb will
+Each time one of your tests asks for a fresh database by calling `pgtestdb.New`, pgtestdb will
 check to see if a template database already exists. If not, it creates a new
-database, runs your migrations on it, and then marks it as a template.  Once the
-template exists, it is _very_ fast to create a new database from that template.
+database, runs your migrations on it, and then marks it as a template. Once the
+template exists, it then creates a test-specific database from that template.
 
-pgtestdb only runs migrations one time when your migrations change. The marginal
-cost of a new test that uses the database is just the time to create a clone
-from the template, which is now basically free.
+Creating a new database from a template is _very_ fast, on the order of 10s of
+milliseconds.  And because pgtestdb uses advisory locks and hashes your
+migrations to determine which template database to use, your migrations only
+end up being run one time, regardless of how many tests or separate packages
+you have. This is true even across test runs --- pgtestdb will only run your
+migrations again if you change them in some way.
 
 When a test succeeds, the database it used is automatically deleted.
 When a test fails, the database it used is left alive, and the test logs will
@@ -334,13 +337,18 @@ config (see below.)
 ```go
 // Config contains the details needed to connect to a postgres server/database.
 type Config struct {
-  DriverName string // "pgx" (pgx) or "postgres" (lib/pq)
-  Host       string //  "localhost"
-  Port       string // "5433"
-  User       string //  "postgres"
-  Password   string // "password"
-  Database   string // "postgres"
-  Options    string // "sslmode=disable&anotherSetting=value"
+	DriverName string // the name of a driver to use when calling sql.Open() to connect to a database, "pgx" (pgx) or "postgres" (lib/pq)
+	Host       string // the host of the database, "localhost"
+	Port       string // the port of the database, "5433"
+	User       string // the user to connect as, "postgres"
+	Password   string // the password to connect with, "password"
+	Database   string // the database to connect to, "postgres"
+	Options    string // URL-formatted additional options to pass in the connection string, "sslmode=disable&something=value"
+	// TestRole is the role used to create and connect to the template database
+	// and each test database. If not provided, defaults to [DefaultRole].  The
+	// capabilities of this role should match the capabilities of the role that
+	// your application uses to connect to its database and run migrations.
+	TestRole *Role
 }
 
 // URL returns a postgres connection string in the format
@@ -357,6 +365,58 @@ new databases and roles. Most likely you want to connect as the default
 `postgres` user, since you'll be connecting to a dedicated testing-only Postgres
 server as described earlier.
 
+### `pgtestdb.Role`
+A dedicated Postgres role (user) is used to create the template database and each test database. pgtestdb will create this role for you with sane defaults, but you can control the username, password, and capabilities of this role if desired.
+
+```go
+const (
+	// DefaultRoleUsername is the default name for the role that is created and
+	// used to create and connect to each test database.
+	DefaultRoleUsername = "pgtdbuser"
+	// DefaultRolePassword is the default password for the role that is created and
+	// used to create and connect to each test database.
+	DefaultRolePassword = "pgtdbpass"
+	// DefaultRoleCapabilities is the default set of capabilities for the role
+	// that is created and used to create and conect to each test database.
+	// This is locked down by default, and will not allow the creation of
+	// extensions.
+	DefaultRoleCapabilities = "NOSUPERUSER NOCREATEDB NOCREATEROLE"
+)
+
+// DefaultRole returns the default Role used to create and connect to the
+// template database and each test database.  It is a function, not a struct, to
+// prevent accidental overriding.
+func DefaultRole() Role {
+	return Role{
+		Username:     DefaultRoleUsername,
+		Password:     DefaultRolePassword,
+		Capabilities: DefaultRoleCapabilities,
+	}
+}
+
+// Role contains the details of a postgres role (user) that will be used
+// when creating and connecting to the template and test databases.
+type Role struct {
+	// The username for the role, defaults to [DefaultRoleUsername].
+	Username string
+	// The password for the role, defaults to [DefaultRolePassword].
+	Password string
+  // The capabilities that will be granted to the role, defaults to
+  // [DefaultRoleCapabilities].
+	Capabilities string
+}
+```  
+
+Because this role is used to connect to each template and each test database
+and run the migrations, its capabilities should match those of your production
+application. For instance, if in production your application connects as a
+superuser, you will want to pass a custom `Role` whthat includes the
+`SUPERUSER` capability so that your migrations will run the same in both
+envproduction and tests.
+
+This is a common case for many applications that install or activate extensions
+like [Postgis](https://postgis.net/), which require activation via a superuser.
+
 ### `pgtestdb.Migrator`
 
 The `Migrator` interface contains all of the logic needed to prepare a template

VERSION

@@ -1 +1 @@
-v0.0.12
+v0.0.13

docker-compose.yml

@@ -1,7 +1,11 @@
 version: "3.6"
 services:
   testdb:
-    image: postgres:15
+    # We're using postgis so that we can test the creation of the postgis
+    # extension, which requires superuser extensions.
+    # 
+    # To use the equivalent in plain postgres, use `postgres:15`
+    image: postgis/postgis:15-master
     environment:
       POSTGRES_PASSWORD: password
     restart: unless-stopped

flake.nix

@@ -29,14 +29,16 @@
               packages = with pkgs; [
                 # Go
                 delve
-                go-outline
                 go
+                go-outline
                 golangci-lint
+                gomodifytags
                 gopkgs
                 gopls
+                gotests
                 gotools
+                impl
                 # Nix
-                rnix-lsp
                 nixpkgs-fmt
                 # Other
                 just

migrators/atlasmigrator/go.mod

@@ -6,7 +6,7 @@ replace github.com/peterldowns/pgtestdb => ../../
 
 require (
 	github.com/jackc/pgx/v5 v5.5.1
-	github.com/peterldowns/pgtestdb v0.0.12
+	github.com/peterldowns/pgtestdb v0.0.13
 	github.com/peterldowns/testy v0.0.1
 )
 

migrators/dbmatemigrator/go.mod

@@ -7,7 +7,7 @@ replace github.com/peterldowns/pgtestdb => ../../
 require (
 	github.com/amacneil/dbmate/v2 v2.4.0
 	github.com/jackc/pgx/v5 v5.5.1
-	github.com/peterldowns/pgtestdb v0.0.12
+	github.com/peterldowns/pgtestdb v0.0.13
 	github.com/peterldowns/testy v0.0.1
 )
 

migrators/golangmigrator/go.mod

@@ -7,7 +7,7 @@ replace github.com/peterldowns/pgtestdb => ../../
 require (
 	github.com/golang-migrate/migrate/v4 v4.16.0
 	github.com/jackc/pgx/v5 v5.5.1
-	github.com/peterldowns/pgtestdb v0.0.12
+	github.com/peterldowns/pgtestdb v0.0.13
 	github.com/peterldowns/testy v0.0.1
 )
 

migrators/goosemigrator/go.mod

@@ -6,7 +6,7 @@ replace github.com/peterldowns/pgtestdb => ../../
 
 require (
 	github.com/jackc/pgx/v5 v5.5.1
-	github.com/peterldowns/pgtestdb v0.0.12
+	github.com/peterldowns/pgtestdb v0.0.13
 	github.com/peterldowns/testy v0.0.1
 	github.com/pressly/goose/v3 v3.11.2
 )

migrators/pgmigrator/go.mod

@@ -7,7 +7,7 @@ replace github.com/peterldowns/pgtestdb => ../../
 require (
 	github.com/jackc/pgx/v5 v5.5.1
 	github.com/peterldowns/pgmigrate v0.0.5
-	github.com/peterldowns/pgtestdb v0.0.12
+	github.com/peterldowns/pgtestdb v0.0.13
 	github.com/peterldowns/testy v0.0.1
 )
 

migrators/sqlmigrator/go.mod

@@ -6,7 +6,7 @@ replace github.com/peterldowns/pgtestdb => ../../
 
 require (
 	github.com/jackc/pgx/v5 v5.5.1
-	github.com/peterldowns/pgtestdb v0.0.12
+	github.com/peterldowns/pgtestdb v0.0.13
 	github.com/peterldowns/testy v0.0.1
 	github.com/rubenv/sql-migrate v1.4.0
 )