Update developer documentation and main readme (#818)

Brings an initial outline on how to start up Monoweb locally, and
limitations externals may have.

---------

Co-authored-by: Brage <[email protected]>

.env.example

@@ -0,0 +1,33 @@
+# Auth0 client configuration, get these from the Auth0 dashboard
+DASHBOARD_AUTH0_CLIENT_ID=
+DASHBOARD_AUTH0_CLIENT_SECRET=
+DASHBOARD_AUTH0_ISSUER=https://dev.id.online.ntnu.no
+WEB_AUTH0_CLIENT_ID=
+WEB_AUTH0_CLIENT_SECRET=
+WEB_AUTH0_ISSUER=https://dev.id.online.ntnu.no
+GTX_AUTH0_CLIENT_ID=
+GTX_AUTH0_CLIENT_SECRET=
+GTX_AUTH0_ISSUER=https://dev.id.online.ntnu.no
+
+# Private secret for signing calendar generation and NextAuth JWTs
+# Generate these yourself with `openssl rand -base64 32` or similar tool
+CAL_KEY=
+NEXTAUTH_SECRET=
+
+# PostgreSQL with pgx-ulid installed
+DATABASE_URL=
+
+# Email API gateway
+EMAIL_ENDPOINT=https://brevduen.staging.online.ntnu.no/integrations/email
+EMAIL_TOKEN=
+
+# Stripe public, secret and webhook secret
+FAGKOM_STRIPE_PUBLIC_KEY=
+FAGKOM_STRIPE_SECRET_KEY=
+FAGKOM_STRIPE_WEBHOOK_SECRET=
+PROKOM_STRIPE_PUBLIC_KEY=
+PROKOM_STRIPE_SECRET_KEY=
+PROKOM_STRIPE_WEBHOOK_SECRET=
+TRIKOM_STRIPE_PUBLIC_KEY=
+TRIKOM_STRIPE_SECRET_KEY=
+TRIKOM_STRIPE_WEBHOOK_SECRET=

CONTRIBUTING.md

@@ -0,0 +1,143 @@
+# Monoweb Developer Guide
+
+This guide is intended for developers who want to contribute to Monoweb. It provides an overview of the project's
+architecture, the tools used, and the local development process.
+
+## Table of Contents
+
+- [Architecture](#architecture)
+- [Tools](#tools)
+- [Local Development](#local-development)
+    - [Required Environment Variables](#required-environment-variables)
+    - [Running with your own PostgreSQL database](#running-with-your-own-postgresql-database)
+
+## Architecture
+
+Monoweb is a monorepo that contains multiple libraries and applications. By application we mean a module or piece of
+software that can be run independently. By library we mean a module, or a piece of software that is used by applications
+or other libraries.
+
+The monorepo is organized as follows:
+
+- `packages/`: Contains all the libraries
+- `apps/`: Contains all the applications
+- `docs/`: Contains the documentation
+- `tools/`: Internal developer tools, such as the Monoweb Database Migrator
+
+We use [PNPM workspaces](https://pnpm.io/workspaces) along with [Turborepo](https://turbo.build/repo/docs) to manage the
+monorepo. All dependencies for all libraries and applications are managed by PNPM.
+
+All of the code in the Monorepo is TypeScript.
+
+<details>
+<summary>How are the libraries packaged and consumed?</summary>
+
+You might wonder how we build the libraries so that the applications can consume them. The answer is that we don't. We
+export the libraries as TypeScript source code, and consume them as TypeScript source code. In the default TypeScript
+compiler, this would pose a problem, since the compiler does not want to enter `node_modules` directories.
+
+However, since we use the Next.js Compiler for the web applications, and TSup for the other applications, we can tell
+the Next.js Compiler or TSup compiler to bundle the libraries as part of the build process.
+
+In Next.js, this is done by adding `transpilePackages` in the `next.config.mjs` files. In Tsup builds, we simply tell it
+to bundle everything into a single file.
+
+Examples of both can be found in the different applications in the `apps/` directory. For example, `apps/web` uses
+Next.js, and `apps/gateway-email` uses TSup.
+</details>
+
+## Tools
+
+The following tools are used to develop Monoweb:
+
+- [Node.js](https://nodejs.org/): JavaScript runtime
+- [PNPM](https://pnpm.io/): Package manager (`npm i -g pnpm`)
+- [Docker](https://www.docker.com/): Containerization for a few applications (not required `apps/web`)
+- [Docker Compose](https://docs.docker.com/compose/): For using local PostgreSQL if wanted (modern versions of Docker
+  include Docker Compose)
+- [Doppler](https://doppler.com/): Secrets management, used for retrieving secrets in development.
+    - If you are not a Dotkom team member, you will not have access to Doppler. You can still run the applications, but
+      you will have to provide the secrets yourself. In this case, you can use a `.env` file in the different
+      application directories, or export them into your shell.
+- [AWS CLIv2](https://aws.amazon.com/cli/): Used for deploying to applications to AWS. You will need to have the correct
+  AWS credentials set up to deploy to AWS. If you are not a dotkom team member, we can unfortunately not provide you
+  with credentials.
+- [Terraform](https://www.terraform.io/): Infrastructure as code, used for managing the infrastructure. Note that the
+  Terraform configuration is located in a separate, private GitHub repository. If you are not a dotkom team member, we
+  can unfortunately not provide you with access to the repository. This is done to prevent any confidential information
+  from being leaked through [Atlantis](https://www.runatlantis.io/).
+
+## Local Development
+
+To get started with local development, ensure you have the [applicable tools](#tools) installed. To build and run all the
+applications, you can use the following commands:
+
+```bash
+doppler login
+doppler setup --project monoweb --config dev
+
+git clone https://github.com/dotkom/monoweb
+cd monoweb
+
+pnpm install
+doppler run pnpm dev
+```
+
+> Note that this, by default, uses a shared database hosted on https://neon.tech. Because this database is shared, you
+> should be careful with what you do with it. If you need to perform migrations or change data, you should set up your
+> branch on Neon. See the [guide for using Neon like a pro](./using-neon-like-a-pro.md) for more information.
+
+If you want to run a specific application, you can use the `--filter` flag:
+
+```bash
+doppler run pnpm dev --filter=@dotkomonline/web
+```
+
+If you have some local variables that you want to use, and override the Doppler ones you can use the `--preserve-env`
+flag:
+
+```bash
+export DATABASE_URL="postgres://<username>:<password>@localhost:5432/<db_name>"
+doppler run --preserve-env pnpm dev
+```
+
+If you are not using Doppler, you need to use a standalone `.env` file placed in the project root. For how to populate the `.env` file, see [this chapter](#required-environment-variables). For how to start local development, follow the rest of the chapter where you omit the use of Doppler:
+
+```diff
+- doppler run pnpm dev
++ pnpm dev
+```
+
+### Required Environment Variables
+
+These are the environment variables that are required to run the applications. Below is a template that you can copy if
+you do not have access to Doppler:
+
+Please consult the example [.env.example](.env.example) file for the environment variables necessary.
+
+### Running with your own PostgreSQL database
+
+> This section requires you to have both [Docker](#tools) and [Docker Compose](#tools) installed.
+
+We use PostgreSQL 15 with the `pgx_ulid` extension. You can use the `packages/pgx-ulid` package to build the extension
+into a Docker image, and then run the image to get a PostgreSQL instance with the extension.
+
+```bash
+cd packages/pgx-ulid
+sh build.sh
+
+# You now have a Docker image named pgx_ulid:0.1.3
+```
+
+If you don't want to manually build the image (it takes a while, and is error prone), you can use Dotkom's publicly
+distributed image. The image is available at `https://gallery.ecr.aws/dotkom/dotkom/pgx-ulid`.
+
+```bash
+docker run -d -p 5432:5432 public.ecr.aws/dotkom/dotkom/pgx-ulid \
+  -e POSTGRES_PASSWORD=postgres \
+  -e POSTGRES_USER=postgres \
+  -e POSTGRES_DB=postgres
+
+# Set your DATABASE_URL to the following:
+export DATABASE_URL="postgres://postgres:postgres@localhost:5432/postgres"
+```

README.md

@@ -1,92 +1,16 @@
-# galactic-thunderdome-x
+# Monoweb
 
-https://owdocs.vercel.app
+Monoweb is the next-generation web application for Online. This monorepo contains all the source code for the
+applications that power the Online web experience.
+
+## Contributing
+
+Please see the [developer guide](docs/developer-guide.md) for information on how to get started with development.
 
 <a href="https://vercel.com?utm_source=[team-name]&utm_campaign=oss" width="150" height="30">
     <img src="https://images.ctfassets.net/e5382hct74si/78Olo8EZRdUlcDUFQvnzG7/fa4cdb6dc04c40fceac194134788a0e2/1618983297-powered-by-vercel.svg" alt="Vercel">
 </a>
 
-## Development  
-
-1. `pnpm i` to install packages
-2. `pnpm build` to build packages
-3. `docker-compose up` to start databases
-4. The command will output a client id, set this in apps/web/.env. Copy the example from .env.example
-5. `pnpm dev` to run the project
-   - You can run `pnpm dev --filter=<project>` to run a certain project from root, such as `pnpm dev --filter=web`
-
-## Environment Setup
-
-The project requires environment variables to run locally. All variables are validated using the env package (located at packages/env/src/env.mjs).
-
-### Internal Tooling
-
-Internally we use Doppler for our secrets management. Send a message to someone in Dotkom or send a mail to `[email protected]` if you believe you should have access.
-
-#### Doppler CLI Setup
-
-1. Install Doppler CLI: Follow the instructions at [Doppler CLI Documentation](https://docs.doppler.com/docs/install-cli)
-2. Authenticate: Run `doppler login`
-3. Configure environment using `doppler setup`
-
-#### Environment Selection
-
-Our setup includes two environments: dev (development) and prd (production).
-
-Use `doppler setup` to choose your environment.
-
-**Important note**: The dev environment connects to a shared database. To avoid annoying other devs using this databsae, avoid performing migrations or changing too much data on this database. Set up your own database instance for such operations:
-
-#### Running with the Shared Database
-
-If you choose to use the shared database in the dev environment:
-
-Run this in the root of the repo:
-`doppler run -- pnpm run dev`
-
-This approach is suitable for general development and testing where database modifications are not required.
-
-#### Running the Project with your own database
-
-```sh
-export DATABASE_URL="postgres://<username>:<password>@0.0.0.0:5432/<db_name>"
-doppler run --preserve-env pnpm dev
-```
-
-### Local Database Setup
-
-The project uses pgx ulid PostgreSQL extension for ULID support. This needs to be installed in your local database.  For convenience, you can use the docker image: public.ecr.aws/z5h0l8j6/dotkom/pgx-ulid:0.1.3 that includes the necessary extension pre installed.
-
-### [Auth0] How to sync user pool with OW
-
-This should not be necessary, as we now create a monoweb user when a user from Auth0 first logs in.
-
-1. Install Auth0 CLI
-2. Log into Auth0 with `auth0 login`
-3. Install PostGreSQL Client (for debian/ubuntu-based: `sudo apt install postgresql-client-14` (or later versions))
-4. Install jq `sudo apt install jq`
-5. Load the database into the shell. (`export DATABASE_URL=$(doppler secrets get DATABASE_URL --plain)`)
-6. Run the command below to sync.
-
-```shell
-# you actually need to paginate...
-auth0 api users \
-  | jq ".[].user_id"
-  | xargs -I '{}' \
-  psql -Atx $DATABASE_URL -c "INSERT INTO ow_user (auth0_sub) VALUES ('{}') ON CONFLICT DO NOTHING"
-```
-
-### [Payment] How to configure stripe (locally)
-
-If you want the payment system to work while running monoweb locally, you will have to follow these steps:
-
-1. Navigate to [this link](https://dashboard.stripe.com/test/webhooks/create?endpoint_location=local) and follow instruction (1) on the page
-2. Follow instruction (2) on the page BUT replace the url in the command with `localhost:3000/api/webhooks/stripe/{public_key}` where {public_key} is your stripe public key
-3. Replace the appropriate webhook secret env variable in .env with the one given by the CLI in the console
-4. Start monoweb with `pnpm dev`
-5. Follow step(3) on the page. This will finish registering your local system as a webhook receipient
-6. Finally, when all steps are finished, click "done"
-
-If you have multiple stripe accounts configured like monoweb has right now, you might want to do this on every account. If not then just make sure that you are using the configured stripe account when you are creating checkout sessions.
+## License
 
-That should be it :)
+Licensed under the MIT license.

docker-compose.yml

@@ -3,11 +3,11 @@ version: "3.2"
 services:
   db:
     container_name: ow_db
-    image: postgres
+    image: public.ecr.aws/dotkom/dotkom/pgx-ulid:0.1.3
     restart: always
     environment:
       POSTGRES_PASSWORD: owpassword123
       POSTGRES_USER: ow
       POSTGRES_DB: ow
     ports:
-      - 6543:5432
+      - "5432:5432"

docs/configure-stripe-locally.md

@@ -0,0 +1,17 @@
+# How to configure stripe locally
+
+If you want the payment system to work while running monoweb locally, you will have to follow these steps:
+
+1. Navigate to [this link](https://dashboard.stripe.com/test/webhooks/create?endpoint_location=local) and follow
+   instruction (1) on the page
+2. Follow instruction (2) on the page BUT replace the url in the command
+   with `localhost:3000/api/webhooks/stripe/{public_key}` where {public_key} is your stripe public key
+3. Replace the appropriate webhook secret env variable in .env with the one given by the CLI in the console
+4. Start monoweb with `pnpm dev`
+5. Follow step(3) on the page. This will finish registering your local system as a webhook receipient
+6. Finally, when all steps are finished, click "done"
+
+If you have multiple stripe accounts configured like monoweb has right now, you might want to do this on every account.
+If not then just make sure that you are using the configured stripe account when you are creating checkout sessions.
+
+That should be it :)

docs/using-neon-like-a-pro.md

@@ -0,0 +1,21 @@
+# Using Neon Like a Pro
+
+This guide will help you get started with the shared Neon database, and the steps you need to take in order to set up
+your own branch on Neon.
+
+Neon offers database branching, which essentially means you create a clone of the primary database for testing or
+development purposes.
+
+> If you are not a dotkom member, you will not have access to the Neon database. You can still run the applications, but
+> you will have to provide the PostgreSQL database yourself.
+
+## Setting up your own branch on Neon
+
+1. Go to [Neon](https://neon.tech) and log in using the `dotbot` account. The email and password combination for this
+   account can be found in Doppler.
+2. Click on the `web-staging` project.
+3. Follow the guide on [Neon Branching](https://neon.tech/docs/introduction/branching)
+
+This provisions a specific compute endpoint for your branch, which you should use for development. Consult the primary
+[developer guide](/CONTRIBUTING.md) for more information on how to override the environment variables (and database
+URL) when running the applications.