Do not reinit db connection locally

[flozia]

Description

When developing locally we reinitialize and create a new database connection each time the server reloads. When hitting your local db client threshold this results in an Unhandled Runtime Error you have likely already had the pleasure to enjoy instead of your latest code changes.

For local development we can just store the initialized db in global and hopefully you’ll have to npm run dev a lot less now.

How to test

Try developing and making changes for a bit on/from this branch. If you are not running into the Error: sorry, too many clients already view or any new issues related to your local database connection all should be good. :)

[pdehaan]

Not sure any of these are relevant to the refactor, but possibly worth verifying:

git grep -n "/knexfile.js" src | cat

src/db/index.js:6:import knexConfig from './knexfile.js'
src/scripts/convertBreachResolutions.js:12:import knexConfig from "../db/knexfile.js";
src/scripts/kube-jobs/convertBreachResolutions.js:12:import knexConfig from "../../db/knexfile.js";
src/scripts/kube-jobs/recencyToBreachId.js:14:import knexConfig from "../../db/knexfile.js";
src/scripts/migrationCleanup.js:11:import knexConfig from "../db/knexfile.js";
src/scripts/prod-experiments/benchmarkNoUpsert.js:12:import knexConfig from "../../db/knexfile.js";
src/scripts/prod-experiments/benchmarkWithUpsert.js:12:import knexConfig from "../../db/knexfile.js";
src/scripts/prod-experiments/pullExperiment100.js:11:import knexConfig from "../../db/knexfile.js";
src/scripts/prod-experiments/pullExperiment1000.js:11:import knexConfig from "../../db/knexfile.js";

Looks like the overwhelming majority of the remaining usages are in src/scripts/**.

[Vinnl]

I think this is also used in our cron jobs and queue scripts, which don't support TypeScript - so it'll probably have to stay .js for now.

(Problems like those are currently non-obvious, which is why I think separate projects with their own specific setup are probably safer.)

[flozia]

Because of that reason I did not make any changes to the scripts that @pdehaan grepped above and only to the relevant files for running the application. But we can also just have this a .js file and replace it everywhere. I was just hesitant to do so because we will likely only notice if anything goes wrong until it is deployed.

Edit: Ok, the changes are still being used in some scripts that import this file. I’ll make this a .js file.

[Vinnl]

Yeah, I figured those scripts possibly import the DB modules we have? But Joe and Rob will know better.

[flozia]

Yup! … and to @pdehaan’s note:

Not sure any of these are relevant to the refactor, but possibly worth verifying:
[…]
Looks like the overwhelming majority of the remaining usages are in src/scripts/**.

Now I can drop this in everywhere: 5d719fa.

[flozia]

@mansaj Do you have any concerns with the db and script related changes?

[rhelmer]

I've been trying this out locally today and haven't noticed the problem with DB connections running out.