Skip to content

Commit

Permalink
docs: top open source postgres migration tools
Browse files Browse the repository at this point in the history
  • Loading branch information
tianzhou committed Jan 16, 2024
1 parent decd601 commit db168f8
Show file tree
Hide file tree
Showing 21 changed files with 117 additions and 11 deletions.
14 changes: 7 additions & 7 deletions content/blog/database-as-code-landscape.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,34 +33,34 @@ In this blog, we will review the state-of-the-art in the Database as Code landsc

Started in 2006, Liquibase is arguably the most well-known product in this segment. When someone asks for database schema change advice on a forum, oftentimes, you will see a reply mentioning Liquibase.

![_](/content/blog/database-as-code-landscape/liquibase-overview.webp)
![_](/images/products/liquibase/liquibase-overview.webp)

Liquibase is both an open source project as well as a company providing its commercial offering. The company used to be called Datical, and was renamed Liquibase to consolidate the branding (wise move).
Liquibase main product is a Java-based CLI. Via the CLI, developer teams can integrate the database schema migration into their CI/CD workflow. For the Java application, Liquibase can also be used as a library. Applications usually embed Liquibase library to apply any applicable schema migration on startup.

In Liquibase, a schema migration unit is encapsulated in a `Change Set`. Probably due to its age and root in Java, the most commonly used form is XML (YAML and JSON support have been added later):

![_](/content/blog/database-as-code-landscape/liquibase-xml.webp)
![_](/images/products/liquibase/liquibase-xml.webp)

Plain SQL is also supported with proper annotation:

![_](/content/blog/database-as-code-landscape/liquibase-sql.webp)
![_](/images/products/liquibase/liquibase-sql.webp)

Liquibase is [migration-based](/blog/database-version-control-state-based-vs-migration-based). It records the incremental change instead of the desired end state of the target database schema.

Recently, Liquibase introduced HUB, it's an information portal for paid customers to view, organize and monitor database change activity in real-time.

![_](/content/blog/database-as-code-landscape/liquibase-hub.webp)
![_](/images/products/liquibase/liquibase-hub.webp)

### Flyway

![_](/content/blog/database-as-code-landscape/flyway-overview.webp)
![_](/images/products/flyway/flyway-overview.webp)

Flyway resembles Liquibase in many ways. It's an open source project with a long history and a large customer base. Its core product includes the CLI and a Java library. Flyway also recently announced a web portal named Hub.

The commercial entity behind Flyway is Redgate through an acquisition. The branding may cause some confusion, while it also sets a boundary between the open source and commercial offering. Flyway carries a more casual branding tone:

![_](/content/blog/database-as-code-landscape/flyway-migration.webp)
![_](/images/products/flyway/flyway-migration.webp)

Flyway website is not as shining as those new DevTools companies using sophisticated gradients. You can even spot some basic UI spacing issues. Yet Flyway shows a classic example that content matters more than cosmetic beauty and its documentation stands out because of the clarity.

Expand All @@ -74,7 +74,7 @@ Sqitch is also an open source project been on the market for a while. Sqitch is

Choosing Perl is not the only interesting aspect. Sqitch has its own design philosophy about how to manage database schema changes. Both Liquibase and Flyway use the file naming convention to control the schema migration behavior (convention over configuration):

![_](/content/blog/database-as-code-landscape/flyway-migration-version.webp)
![_](/images/products/flyway/flyway-migration-version.webp)

While Sqitch takes an explicit approach. In the below example, you need to name the schema migration as `appschema`:

Expand Down
1 change: 1 addition & 0 deletions content/blog/top-database-schema-change-tool-evolution.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,3 +147,4 @@ for free or [start locally with a single command](/docs/get-started/self-host/#d

- [Top Postgres Extensions](/blog/top-postgres-extension/)
- [Top Postgres GUI Clients](/blog/top-postgres-gui-client/)
- [Top Open Source Postgres Migration Tools](/blog/top-open-source-postgres-migration-tools/)
107 changes: 107 additions & 0 deletions content/blog/top-open-source-postgres-migration-tools.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---
title: Top Open Source Postgres Migration Tools in 2024
author: Tianzhou
published_at: 2024/01/16 09:00:00
feature_image: /content/blog/top-open-source-postgres-migration-tools/cover.webp
tags: Industry
featured: true
description: What are the top open source Postgres schema migration tools.
---

PostgreSQL just won DBMS of the Year 2023 and is enjoying a vibrant ecosystem. On the other hand,
database schema migration in Postgres is still PITA. In this post, we are reviewing several open source
database schema migration tools for Postgres.

[![starhistory](/content/blog/top-open-source-postgres-migration-tools/star-history.webp)](https://star-history.com/#bytebase/bytebase&xataio/pgroll&liquibase/liquibase&flyway/flyway&graphile/migrate&fabianlindfors/reshape&Date)

## Bytebase

[Bytebases](https://github.com/bytebase/bytebase) is an all-in-one database DevOps and CI/CD solution. It's like GitHub/GitLab that provides
a GUI workspace for developers and DBAs to collaborate on database changes. It's written in Go and TypeScript.

![issue-detail](/content/blog/top-open-source-postgres-migration-tools/issue-detail.webp)

It also provides configurable SQL lint rules to detect SQL anti-patterns such as `NOT NULL` enforcement.

![sql-review](/content/blog/top-open-source-postgres-migration-tools/sql-review.webp)

Apart from taking care of database change management, Bytebase also provides a SQL Editor with data
access control, dynamic data masking and audit logs to control the query path.

![sql-editor](/content/blog/top-open-source-postgres-migration-tools/sql-editor.webp)

## Liquibase

[Liquibase](https://github.com/liquibase/liquibase) is arguably the most well-known product in this segment. It's CLI-based and written in Java.

In Liquibase, the schema migrations are organized as `Changeset` and `Changelog`. Probably due to its age and root in Java, the most commonly used form is XML (YAML and JSON support have been added later):

![_](/images/products/liquibase/liquibase-xml.webp)

Plain SQL is also supported with proper annotation:

![_](/images/products/liquibase/liquibase-sql.webp)

## Flyway

[Flyway](https://github.com/flyway/flyway) is another open source project with a long history and a large customer base. Its core product includes the CLI and a Java library.

Flyway has been acquired by Redgate, while it still retains a developer-oriented approach. This can be
felt from its website.

![_](/images/products/flyway/flyway-migration.webp)

Liquibase and Flyway are closely matched in their offerings. The key distinction between them is their target audiences: Liquibase is oriented towards serving enterprise clients, whereas Flyway presents itself as more developer-friendly and approachable.

## Reshape

[Reshape](https://github.com/fabianlindfors/reshape) takes a novel approach to achieve zero-downtime
schema migrations. It's CLI based and written in Rust. Reshape schema migration consists of 3 phases:

1. **Start migration** (`reshape migration start`): Sets up views and triggers to ensure both the new and old schema are usable at the same time.

1. **Roll out application**: Your application can be gradually rolled out without downtime. The existing deployment will continue using the old schema whilst the new deployment uses the new schema.

1. **Complete migration** (`reshape migration complete`): Removes the old schema and any intermediate data and triggers.

The author is now working on [ReshapeDB](https://reshapedb.com/), a new database built from the ground up to make zero-downtime schema and data migrations as simple and safe as possible.

![_](/images/products/reshape/reshape.webp)

FWIW, [pgroll](https://github.com/xataio/pgroll) is another Postgres schema
migration tool inspired by Reshape. pgroll is CLI based and written in Go. It's built by [Xata](https://xata.io/), the serverless database platform built on the top of PostgreSQL.

![_](/images/products/pgroll/pgroll-migration-flow.webp)

## graphile-migrate

[graphile-migrate](https://github.com/graphile/migrate) is an [opinionated](https://github.com/graphile/migrate?tab=readme-ov-file#opinions) SQL-powered productive roll-forward migration tool for PostgreSQL.

graphile-migrate is also CLI-based, written in TypeScript. It can also work with [PostGraphile](https://www.graphile.org/postgraphile/) to generate GraphQL schemas upon database changes.

Another novel idea graphile-migrate introduces is the `commit`, `uncommit` commands. It enables a
git-like workflow when developing against the local database.

- `graphile-migrate commit` commits the current migration into the `committed/` folder, resetting the
current migration. Resets the shadow database.

- `graphile-migrate uncommit` Moves the latest commit out of the committed migrations folder and back to the current migration
(assuming the current migration is empty-ish). Removes the migration tracking entry from **ONLY** the local database.

## Summary

| Name | Interface | Language | Postgres Only | Highlight |
| ------------------------------------------------------- | --------- | --------------- | ------------- | ------------------------------------------------------------------ |
| [Bytebase](https://github.com/bytebase/bytebase) | GUI | Go + TypeScript | N | All in one for team collaboration. |
| [Liquibase](https://github.com/liquibase/liquibase) | CLI | Java | N | Long-standing. |
| [Flyway](https://github.com/flyway/flyway) | CLI | Java | N | Developer-friendly. |
| [Reshape](https://github.com/fabianlindfors/reshape) | CLI | Rust | Y | Zero-downtime based on view. |
| [pgroll](https://github.com/xataio/pgroll) | CLI | Go | Y | Inspired by Reshape and more accessible via its database platform. |
| [graphile-migrate](https://github.com/graphile/migrate) | CLI | TypeScript | Y | Opinionated with git-like workflow. |

## Further Readings

- [Top Postgres Extensions](/blog/top-postgres-extension/)
- [Top Postgres GUI Clients](/blog/top-postgres-gui-client/)
- [Postgres vs. MySQL](/blog/postgres-vs-mysql)
- [Postgres vs. MongoDB](/blog/postgres-vs-mongodb)
2 changes: 1 addition & 1 deletion content/blog/top-postgres-extension.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,6 @@ processing, then Postgres is the only choice thanks to the PostGIS extension. Fo
## Further Readings

- [Top Postgres GUI client](/blog/top-postgres-gui-client)
- [Top Database Schema Migration Tools](/blog/top-database-schema-change-tool-evolution/)
- [Top Open Source Postgres Migration Tools](/blog/top-open-source-postgres-migration-tools)
- [Postgres vs. MySQL](/blog/postgres-vs-mysql)
- [Postgres vs. MongoDB](/blog/postgres-vs-mongodb)
4 changes: 1 addition & 3 deletions content/blog/top-postgres-gui-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,7 @@ It is crucial to carefully select the appropriate option that aligns with your o

## Futher Readings

- [Top Database Schema Migration Tools](/blog/top-database-schema-change-tool-evolution/)
- [Top Open Source SQL Clients](/blog/top-open-source-sql-clients/)
- [Top DBeaver Alternatives](/blog/top-dbeaver-alternative)
- [Top Open Source Postgres Migration Tools](/blog/top-open-source-postgres-migration-tools/)
- [Top Postgres Extension](/blog/top-postgres-extension)
- [Postgres vs. MySQL](/blog/postgres-vs-mysql)
- [Postgres vs. MongoDB](/blog/postgres-vs-mongodb)
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added public/images/products/reshape/reshape.webp
Binary file not shown.

1 comment on commit db168f8

@vercel
Copy link

@vercel vercel bot commented on db168f8 Jan 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.