Skip to content

Commit

Permalink
PR review and edits the the "Add a database" section
Browse files Browse the repository at this point in the history
Nathan, please review my changes for correctness. Also, please search for NATHAN in the milestone1-step1.md file and address the two issues/questions I call out.
  • Loading branch information
kevin-atx committed Apr 10, 2024
1 parent dd49260 commit 0df3970
Show file tree
Hide file tree
Showing 6 changed files with 55 additions and 49 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@ title: Overview

Liquibase is designed to be cross-database and separates the database-specific logic from the overall database-agnostic code.

In general, Liquibase works against standard SQL and JDBC calls so most functionality works out of the box.
Therefore, the overall process of adding support for a new database is finding and fixing the specific places where the default logic
doesn't work with your database.
Liquibase works with standard SQL and JDBC calls so most new database extension functionality works out of the box.

This work is broken up into two milestones:
The overall process of adding support for a new database is focused on finding and fixing the specific places where the default logic doesn't work for your database.

1. [Foundational Support](#foundational) which enables Liquibase to understand your database and get basic update/rollback logic working
2. [Advanced Support](#advanced) which enables "change type" and snapshot-based functionality
This extension guide is broken up into two milestones:

In each milestone, there is an end goal of working functionality and have a specific subset of interfaces to implement.
1. [Foundational Support](#foundational-support) which enables Liquibase to understand your database and get basic update/rollback logic working
2. [Advanced Support](#advanced-support) which enables "change type" and snapshot-based functionality

Each milestone has a goal of working functionality using a specific subset of interfaces.

!!! note

Expand All @@ -24,39 +24,38 @@ In each milestone, there is an end goal of working functionality and have a spec
NoSQL and non-relational databases use the same process, but there there is less default behavior to rely on
and so there will be more changes to make.

If you are interested in adding support for a NoSQL database, contact [the Liquibase team](mailto:kevin@liquibase.com) for more information.
If you are interested in adding support for a NoSQL database, contact [the Liquibase team](mailto:community@liquibase.com) for more information.

## Prerequisites

Implementing support for additional databases requires an understanding of Java.
You will be creating classes, overriding methods, and working with inheritance hierarchies.

You will also need to understand how to work with the new database.
As you hit places where Liquibase incorrectly assumes particular SQL will work against your database, you will need to know what the correct SQL is.
You will also need to understand how to work with the new database. As you find situations where Liquibase incorrectly assumes particular SQL will work for your database, you will need to know what the correct SQL needs to be.

## Project Setup

If you have not already created a repository to hold your code, see [Your First Extension](../../extensions-overview/your-first-extension.md) in the Getting Started guide.

## <a name="foundational"></a>Foundational Support
## Foundational Support

The first milestone is to have Liquibase understand how your database works and to be able to run any functionality that depends on user-specified SQL.

The first milestone is to have Liquibase understand how your database works and be able to run any functionality that depends on user-specified SQL.
You will be able to run update and rollback commands using a [Formatted SQL](https://docs.liquibase.com/concepts/changelogs/sql-format.html){:target="_blank"} changelog or you can use the [`sql`](https://docs.liquibase.com/change-types/sql.html){:target="_blank"} change type in a xml/yaml/json changelog.

Basically "you can run update and rollback using [Formatted SQL](https://docs.liquibase.com/concepts/changelogs/sql-format.html){:target="_blank"} or
the [sql change type](https://docs.liquibase.com/change-types/sql.html){:target="_blank"} in xml/yaml/json.
You will not be able to use more complex change types like `createTable` or use `snapshot` or `generateChangelog` functionality until you have finished [milestone 2](#advanced)
You will not be able to use more complex change types like `createTable`. You will also not be able to use `snapshot` or `generateChangelog` commands until you have finished [milestone 2](#advanced-support)

There are three steps in this milestone:

1. [Create a new `liquibase.database.Database` implementation](milestone1-step1.md)
2. [Test that Liquibase works with sql-based changelog files](milestone1-step2.md)
3. [(If step 2 fails) Test and fix failures until there are no more](milestone1-step3.md)

## <a name="advanced"></a>Advanced Support
## Advanced Support

At the end of this second milestone, Liquibase works against your database in a large variety of use cases, but it does not support everything Liquibase can do.
At the end of this second milestone, Liquibase will work with your database in a large number of use cases, but it does not support everything Liquibase can do.

By default, the change and snapshot logic uses standard SQL and/or JDBC calls so **_many_** will work out of the box, but not enough to advertise support for them until you have ensured they work.
By default, the change and snapshot logic uses standard SQL and/or JDBC calls so **_many_** will work out of the box, but not enough to advertise support for them until you have validated they work.

The steps in this phase are:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ title: "Milestone 1: New Database"

## Overview

When adding support for a new database, the first class to create is a new [liquibase.database.Database](../../../code/api/database-database.md) implementation.
This class acts as the "dialect" definition and the facade to your database.
When adding support for a new database, the first class to create is a new [liquibase.database.Database](../../../code/api/database-database.md) implementation. This class acts as the "dialect" definition and the facade to your database.

The Database interface defines methods for:

Expand All @@ -19,8 +18,8 @@ The Database interface defines methods for:

If your database generally attempts to be compatible with another database, your new Database implementation can extend the existing Database class.

For example, many databases strive for PostgreSQL compatibility and so can extend [liquibase.database.core.PostgresDatabase](https://javadocs.liquibase.com/liquibase-core/liquibase/database/core/PostgresDatabase.html){:target="_blank"}
and only override what is special about your database.
For example, many databases strive for PostgreSQL compatibility. Those databases can extend [`liquibase.database.core.PostgresDatabase`](https://javadocs.liquibase.com/liquibase-core/liquibase/database/core/PostgresDatabase.html){:target="_blank"}
and only override what is special about the new database.

!!! note
To correctly differentiate your database from the "standard" version, you must always implement:
Expand All @@ -29,7 +28,9 @@ and only override what is special about your database.
- getShortName()
- getDatabaseProductName()

### Exaample Code
NATHAN - please give guidance on what values to use for the above 3 methods.

### Example Code

```java
--8<-- "src/main/java/com/example/database/ExamplePostgresDatabase.java"
Expand All @@ -42,6 +43,8 @@ which provides default logic that follows SQL standards.

Depending on your base class you will have more or less abstract methods which must be implemented.

NATHAN - could you clarify the above sentence? The statement "you will have more or less abstract methods" is unclear.

### Example Code

```java
Expand All @@ -50,7 +53,7 @@ Depending on your base class you will have more or less abstract methods which m

## API Documentation

A complete description of the API, including what methods must be implemented and how is available [on the liquibase.database.Database API page](../../../code/api/database-database.md).
A complete description of the API, including what methods must be implemented and how, is available on the [liquibase.database.Database API page](../../../code/api/database-database.md).

## Next Step

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: "Milestone 1: Test"

## Overview

With your [new Database class defined](milestone1-step1.md), Liquibase should be able to run user-defined SQL statements against the database. Give it a try!
With your [new Database class defined](milestone1-step1.md), Liquibase should be able to run user-defined SQL statements on the database. Give it a try!

## Testing

Expand All @@ -24,7 +24,7 @@ create table company (id int not null primary key, name varchar(255))
--rollback drop table company
```

and run `liquibase update` against your database using that changelog file.
and run `liquibase update` on your database using that changelog file.

If there are any failures with creating the databasechangelog table, managing the lock, marking the change sets ran, or anything else. [GOTO step 3](milestone1-step3.md).

Expand All @@ -38,12 +38,13 @@ Once `update` works, any Liquibase functionality that doesn't rely on modeled ch

## Next Steps

If/when you run into problems when testing, [go to step 3](milestone1-step3.md).
If/when you run into problems during testing, [go to step 3](milestone1-step3.md).

If everything seems to work, Congratulations!! :material-party-popper: Now is a great time to release the first version of your extension.
After releasing, you can move on to [Milestone 2](milestone2-step1.md) to add advanced support.
After releasing, you can proceed to [Milestone 2](milestone2-step1.md) to add advanced support.

!!! tip

If you'd like your database extension to be included in the main Liquibase documentation and be a more official part of Liquibase, [contact the Liquibase team](mailto:[email protected]) and we can work
with you to bring it more officially into the fold.
If you'd like your database extension to be included in the main Liquibase documentation and be a more official part of Liquibase,
[contact the Liquibase team](mailto:[email protected]) and we will work with you to make it more visible in documentation
and in the supported databases list.
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,27 @@ title: "Milestone 1: Fix & Retest"

The functionality you [tested for milestone 1](milestone1-step2.md) relies on:

- Create and populate `databasechangeloglock` table
- Create and populate `databasechangelog` table
- Create and populate the `databasechangeloglock` table
- Create and populate the `databasechangelog` table
- Execute user-specified SQL

## Improving Your Database Support

The standard [liquibase.changelog.ChangeLogHistoryService](https://javadocs.liquibase.com/liquibase-core/liquibase/changelog/ChangeLogHistoryService.html){:target="_blank"},
[liquibase.lockservice.LockService](https://javadocs.liquibase.com/liquibase-core/liquibase/lockservice/LockService.html){:target="_blank"}, and [liquibase.executor.Executor](https://javadocs.liquibase.com/liquibase-core/liquibase/executor/Executor.html){:target="_blank"}
implementations which drive that rely on SQL and/or JDBC standards guided by dialect info from your `Database` class.
implementations rely on SQL and/or JDBC standards guided by dialect information from your `Database` class.

If you run into any problems with those standard services, the most likely fix is overriding additional `liquibase.database.Database` methods to more correctly
describe how your database works.
If you run into any problems with those standard services, the most likely fix is to override additional `liquibase.database.Database` methods to more correctly describe how your database works.

For example, if the `databasechangeloglock` table isn't being created correctly because your database quotes object names in a special way, override the `escapeObjectName` function.

The full list of methods you can override can be found in [liquibase.database.Database API](https://javadocs.liquibase.com/liquibase-core/liquibase/database/Database.html){:target="_blank"}.

!!! tip

Exactly what you need to override will depend on the actual problems you hit. If you have questions on what it takes to fix your problem, ask [on the forum](https://forum.liquibase.org){:target="_blank"}.
Exactly what you need to override will depend on the actual problems you hit. If you have questions on what
it takes to fix your problem, ask [on the forum](https://forum.liquibase.org){:target="_blank"} or
[contact the Liquibase team](mailto:[email protected]).

## Advanced Fixes

Expand All @@ -43,14 +44,14 @@ Instead, you may need to [override one or more](../add-a-sql-generator.md) of th

## Iterate

After you have made a potential fix to the problem you found, re-run your test and see if Liquibase is working better.
After you make a potential fix to the problem you found, re-run your test and see if Liquibase is working better.

- Sometimes your fix didn't work as well as you hoped. Try a new one
- Sometimes your fix resolved one issue, but now you have hit another. Fix that up
- Eventually, you will be done. Congratulations!! :material-party-popper:

## Next Steps

With Liquibase working against your database, now is a great time to release the first version of your extension and get feedback.
With Liquibase working for your database, now is a great time to release the first version of your extension and get feedback.

When you are ready to build advanced support, you can move on to [milestone 2](milestone2-step1.md)
When you are ready to build advanced support, you can start working on [milestone 2](milestone2-step1.md)
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ title: "Milestone 2: Advanced Tests"

As you move beyond [foundational support](milestone1-step1.md), the testing requirements become more complex.

Ensuring snapshot and change types work requires going through each specific change type and through each specific object type, then ensuring they execute correctly for all the different argument permutations that are valid for your database.
Ensuring the `snapshot` command and change types work requires testing each change type and object type to verify that they execute correctly for the different argument permutations that are valid for your database.

To automate this process, Liquibase provides a [Test Harness](https://github.com/liquibase/liquibase-test-harness){:target="_blank"} to help you find everywhere the default logic isn't compatible with your database.
To automate this process, Liquibase provides a [Test Harness](https://github.com/liquibase/liquibase-test-harness){:target="_blank"} to help you find situations where the default logic isn't compatible with your database.

## Test Harness Configuration

Expand All @@ -22,7 +22,7 @@ When completed, you should have a `harness-config.yml` file that specifies how t

The [Test Harness 'Framework' documentation](https://github.com/liquibase/liquibase-test-harness#framework){:target="_blank"} describes how the tests work and how to run them.

At the most basic level, you can run the tests using `mvn test` using the configuration options defined in the test-harness framework documentation such as `-DchangeObjects`.
At the most basic level, you can run the tests using `mvn test` with the configuration options defined in the test-harness framework documentation such as `-DchangeObjects`.
If you are using an IDE, you can run your `ExtensionHarnessTest` class directly and pass along the same settings as system properties.

## Next Step
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,23 +23,24 @@ When searching for a file such as `file.ext`, Test Harness checks for the follow

where it uses the contents of file it finds.

This means that when defining "expectedSql" or changelogs or expectedSnapshots or any other file test harness expects, you can override the default expectations by creating new files higher up in the pattern.
This means that when defining "expectedSql", changelogs, expectedSnapshots or any other file test harness expects, you can override the default by creating new files higher up in the pattern.

!!! tip

Keep your files as generic as possible. If the expectedSql file works for all versions of your database, create the file in ${base directory}/${db name}/file.ext`.
Keep your files as generic as possible. If the expectedSql file works for all versions of your database,
create the file in ${base directory}/${db name}/file.ext`.

Only use major and minor versions when it actually depends on the specific versions.

## Golden Master Files

Many of the tests use "expected" files and will either auto-create them if they do not exist or fail until the file is created.

These files let you manually inspect the SQL being run by a particular change setup to ensure it is doing what you expect now, and will be checked into Git so that it is easy to detect any changes to this known good behavior down the road.
These files let you manually inspect the SQL being run by a particular change type to ensure it is doing what you expect. The completed files should be checked into Git so it is easy to detect any changes to this known good behavior in the future.

## Fixing Logic

For tests that are failing because how Liquibase interacts with the database needs to be customized for your database, you will create one of two types of classes:
For tests that are failing because of Liquibase interactions that need to be customized for your database, you will create one of two types of classes:

- [New SqlGenerator implementations](../add-a-sql-generator.md) for Change -> SQL mappings that are wrong
- [New SnapshotGenerator implementations](../add-a-snapshot-generator.md) for snapshot logic that is wrong
Expand All @@ -48,15 +49,15 @@ The general pattern for both is to create a new class that returns a higher prio

!!! tip

What method you implement to specify the Database implementation your new generator classes are for will depend on the specific class you are implementing.
The Database method you will need to implement for your new generator classes will depend on the specific class you are implementing.

See the corresponding documentation pages for more information and examples.

## Iterate

The first time you run the tests, you will likely get many failures, but don't be discouraged. Often times the first few fixes you make will resolve many of the failures.
The first time you run the tests, you will likely get many failures, but don't be discouraged. Often, the first few fixes you make will resolve many of the failures.

After you have made a potential fix, re-run the tests to ensure it solved the problem and then move on to the next failure.
After making a potential fix, re-run the tests to ensure it solved the problem and then move on to the next failure.
You should quickly pick up the general code patterns, and most of the effort is finding the correct SQL syntax for your database.

!!! tip
Expand All @@ -66,6 +67,7 @@ You should quickly pick up the general code patterns, and most of the effort is
## Complete

Once all the Test Harness tests are passing, you will be able to use any of the standard change types in XML/YAML/JSON changelogs as well as any snapshot-based functionality.

For example, all of this should now work:

- `liquibase update` with `<createTable...` in the changelog
Expand Down

0 comments on commit 0df3970

Please sign in to comment.