Example Go Rest API with a PostgreSQL database

This repository demonstrates how to fully wire up and deploy a Golang REST API with a PostgreSQL database backend.

The application demonstrates:

• REST API using Echo
• PostgreSQL integration using PGX
• Database migrations using Atlas

Use as a Template

If you want to use this as a template, here a short guide:

1. Modify the go.mod file module to be the name of your project
2. Modify the database/schema.hcl file to meet your database needs
3. Modify the docker-compose.yml file to use the name of your project
4. Modify pkg/common/router/router and add additional routes as needed
5. Add your domain models to pkg/common/models directory
6. Add your data access (repository) to the pkg/common/db directory

In general, when you introduce a new separate domain concept:

1. Update the database/schema.hcl file with new tables
2. Create a new route handler pkg/common/handler/xxx.go
3. Create model types pkg/common/models/xxx.go
4. Create a new repository pkg/common/db/xxx.go
5. Update the main pkg/common/router/router.go file to use your new routes / handlers
6. Add a new pkg/common/router/xxx_test.go file to test your new routes / handlers

If you want to add additional config entries:

1. Modify the Config struct in pkg/common/config.go
2. Add a default (typically works locally) value in cmd/config.yaml
3. Override the default value through an environment variable at runtime (i.e. not local) as needed

Technology Choices

The author of this repository carries opinions about how to best organize an application / repository. Those opinions / choices are described below:

1. No ORM (Object Relational Mapper) - Golang has a few ORMs that are popular, in particular GORM and Ent. However, the mapping ability in tools like Scany take care of a lot of tedious work of working with databases (mapping rows to structs). Declaritive SQL in code (imo) is preferable to code generation and clunky SQL-esque APIs.
2. Atlas for database migrations. There are 1000 ways to run migrations in golang, and unfortunately there doesn't seem to be a lot of consensus. However, what I enjoy about Atlas is that it includes Terraform support and it allows you to define your schema in an HCL file and it magically figures out how to update your target database. You can see this in database/schema.hcl and just a simple atlas schema apply and your database is now in sync.
3. Echo for the REST API - There are a lot of http frameworks in the golang echosystem, including Echo and Gin. Honestly, I found using the docs with Echo simpler, but don't have a strong opinion on the web framework side. net/http might be just as good as the other choices, Gin being pretty popular.
4. PGX for database aaccess - database/sql would be fine as well, but PGX is PostgreSQL optimized and is a good choice when tied to Postgres.
5. Taskfile instead of Makefile - There is nothing inherently wrong with Makefile, Taskfile is a reasonable alternative with simple, validatable YAML syntax
6. Pre-commit - makes sure that users cannot commit / push code that isn't up to standards. In this project we check formatting, linting, golang critic, among others to keep the repo tidy.

Pre-requisites

1. Install Docker. Used for testing
2. Install Taskfile, required to do pretty much anything
3. Install Pre-Commit. This project uses pre-commit to ensure code is all nice and tidy before others can see it.
4. Install the pre-commit hooks by running pre-commit install
5. Optionally install Atlas. Atlas is used for database migrations. Note: you can skip this step and just rely on docker, as atlas is only needed to explore its abilities

Quick Start

Make sure you have Docker and Taskfile installed...

1. Run task server.run
   1. Starts up the postgres docker database
   2. Runs migrations so the database is ready
   3. Performs a build of the REST API
   4. Starts the REST API on port 1323. Access on http://localhost:1323

Project Structure

• cmd - this is where the default config and the main app lives
• database - this is where the schema.hcl file lives. Modify this file to alter the database
• pkg - this is where most of the code lives
  • common
    • config - for loading the config file / incorporating environment variable overrides
    • db - the underlying database in PGX, and domain specific Repositories
    • handler - for handling each of the type of echo requests / routes
    • models - core domain model classes, surfaced in the API and used in the Repositories
    • router - where we map echo routes to handlers

Running tasks

This project uses Taskfile for running tasks. The following tasks are available

• build - Builds a local executable, outputs to out/bin/gomin
• clean - Cleans up build artifacts, including out, bin, and test reports
• d.build - Builds the docker iamge, marks it as latest
• d.down - Shuts down all docker containers in the docker compose file
• d.up - Starts up all docker containers, builds and runs the API as well
• db.migrate - Runs the database migration, ensures that the local postgres database is running
• db.up - Starts the database WITHOUT migrations
• server.run - Starts the database, runs migrations, builds the server, and starts the server
• test - Runs all of the tests in all of the go source directories

Writing Tests

This project primarily uses E2E tests hitting the API directly and using the docker postres database.

Tests live in pkg/common/router - create a new xxx_test.go file as needed

Environment Variables and Config

This project is setup to load a default config found in cmd/config.yaml that is overridable via Environment Variables. For example, you can override the database url by setting an environment variable named APP_DB_URL.

Roadmap

1. - Simple REST API
2. - Add initial Makefile
3. - Add pre-commit
4. - Initial database setup
5. - Incorpoate database into REST API
6. - Integration tests for database
7. - E2E tests for REST API
8. - Add github build for golang
9. - Add docker packaging
10. - Add docker image build in Github Actions on Tag
11. - Terraform plan for RDS
12. - Terraform plan for ECS
13. - Github Actions to run terraform and deploy to AWS