Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(drizzle): allow to customize the drizzle schema with before / after init hooks #8196

Merged
merged 25 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
cba157e
feat(drizzle): allow to customize the db schema with before / after i…
r1tsuu Sep 12, 2024
4951ad9
chore: fix types
r1tsuu Sep 12, 2024
a137c99
chore: start tests
r1tsuu Sep 12, 2024
c98b897
dropDatabase for schema hooks test before
r1tsuu Sep 13, 2024
ef783e6
Merge branch 'beta' of github.com:payloadcms/payload into feat/drizzl…
r1tsuu Sep 13, 2024
ffb083a
chore: hooks tests
r1tsuu Sep 13, 2024
dd1ed78
fix lint
r1tsuu Sep 13, 2024
c9466ef
chore: revert jest setup
r1tsuu Sep 13, 2024
470d00c
chore: fix sqlite test
r1tsuu Sep 13, 2024
fa9b0da
fix: sqlite types record, replace with args object, fix typo in test
r1tsuu Sep 14, 2024
6f151f0
chore: explict return type
r1tsuu Sep 15, 2024
8f33b0f
Merge branch 'beta' of github.com:payloadcms/payload into feat/drizzl…
r1tsuu Sep 15, 2024
411c176
feat: extendDrizzleTable helper
r1tsuu Sep 15, 2024
7407364
chore: example for composite unique
r1tsuu Sep 15, 2024
fb7f3ea
chore: add tests for extra column and composite unique
r1tsuu Sep 15, 2024
61a1bd7
chore: fix build / types
r1tsuu Sep 16, 2024
257bc90
chore: add postgres / sqlite docs, fix types / tests
r1tsuu Sep 16, 2024
44d9537
chore: revert jest setup changes
r1tsuu Sep 16, 2024
6082424
chore: fix docs postgresAdapter -> sqliteAdapter
r1tsuu Sep 16, 2024
0a8b946
Merge branch 'beta' of github.com:payloadcms/payload into feat/drizzl…
r1tsuu Sep 16, 2024
a586022
Merge branch 'beta' of github.com:payloadcms/payload into feat/drizzl…
r1tsuu Sep 17, 2024
5fa0e28
chore: comment
r1tsuu Sep 17, 2024
7811395
Merge branch 'beta' of github.com:payloadcms/payload into feat/drizzl…
r1tsuu Sep 17, 2024
befd77a
reset diff
r1tsuu Sep 17, 2024
e121fae
Merge branch 'beta' of github.com:payloadcms/payload into feat/drizzl…
r1tsuu Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 133 additions & 0 deletions docs/database/postgres.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ export default buildConfig({
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |
| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |

## Access to Drizzle

Expand Down Expand Up @@ -97,3 +99,134 @@ Alternatively, you can disable `push` and rely solely on migrations to keep your
In Postgres, migrations are a fundamental aspect of working with Payload and you should become familiar with how they work.

For more information about migrations, [click here](/docs/beta/database/migrations#when-to-run-migrations).

## Drizzle schema hooks

### beforeSchemaInit

Runs before the schema is built. You can use this hook to extend your database structure with tables that won't be managed by Payload.

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { integer, pgTable, serial } from 'drizzle-orm/pg-core'

postgresAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
return {
...schema,
tables: {
...schema.tables,
addedTable: pgTable('added_table', {
id: serial('id').notNull(),
}),
},
}
},
],
})
```

One use case is preserving your existing database structure when migrating to Payload. By default, Payload drops the current database schema, which may not be desirable in this scenario.
To quickly generate the Drizzle schema from your database you can use [Drizzle Introspection](https://orm.drizzle.team/kit-docs/commands#introspect--pull)
You should get the `schema.ts` file which may look like this:

```ts
import { pgTable, uniqueIndex, serial, varchar, text } from 'drizzle-orm/pg-core'

export const users = pgTable('users', {
id: serial('id').primaryKey(),
fullName: text('full_name'),
phone: varchar('phone', { length: 256 }),
})

export const countries = pgTable(
'countries',
{
id: serial('id').primaryKey(),
name: varchar('name', { length: 256 }),
},
(countries) => {
return {
nameIndex: uniqueIndex('name_idx').on(countries.name),
}
},
)

```

You can import them into your config and append to the schema with the `beforeSchemaInit` hook like this:

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { users, countries } from '../drizzle/schema'

postgresAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
return {
...schema,
tables: {
...schema.tables,
users,
countries
},
}
},
],
})
```

Make sure Payload doesn't overlap table names with its collections. For example, if you already have a collection with slug "users", you should either change the slug or `dbName` to change the table name for this collection.


### afterSchemaInit

Runs after the Drizzle schema is built. You can use this hook to modify the schema with features that aren't supported by Payload, or if you want to add a column that you don't want to be in the Payload config.
To extend a table, Payload exposes `extendTable` utillity to the args. You can refer to the [Drizzle documentation](https://orm.drizzle.team/docs/sql-schema-declaration).
The following example adds the `extra_integer_column` column and a composite index on `country` and `city` columns.

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { index, integer } from 'drizzle-orm/pg-core'
import { buildConfig } from 'payload'

export default buildConfig({
collections: [
{
slug: 'places',
fields: [
{
name: 'country',
type: 'text',
},
{
name: 'city',
type: 'text',
},
],
},
],
db: postgresAdapter({
afterSchemaInit: [
({ schema, extendTable, adapter }) => {
extendTable({
table: schema.tables.places,
columns: {
extraIntegerColumn: integer('extra_integer_column'),
},
extraConfig: (table) => ({
country_city_composite_index: index('country_city_composite_index').on(
table.country,
table.city,
),
}),
})

return schema
},
],
}),
})

```
137 changes: 134 additions & 3 deletions docs/database/sqlite.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ export default buildConfig({
## Options

| Option | Description |
|-----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `client` \* | [Client connection options](https://orm.drizzle.team/docs/get-started-sqlite#turso) that will be passed to `createClient` from `@libsql/client`. |
| `push` | Disable Drizzle's [`db push`](https://orm.drizzle.team/kit-docs/overview#prototyping-with-db-push) in development mode. By default, `push` is enabled for development mode only. |
| `migrationDir` | Customize the directory that migrations are stored. |
Expand All @@ -44,8 +44,8 @@ export default buildConfig({
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |


| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |

## Access to Drizzle

Expand Down Expand Up @@ -79,3 +79,134 @@ Alternatively, you can disable `push` and rely solely on migrations to keep your
In SQLite, migrations are a fundamental aspect of working with Payload and you should become familiar with how they work.

For more information about migrations, [click here](/docs/beta/database/migrations#when-to-run-migrations).

## Drizzle schema hooks

### beforeSchemaInit

Runs before the schema is built. You can use this hook to extend your database structure with tables that won't be managed by Payload.

```ts
import { sqliteAdapter } from '@payloadcms/db-sqlite'
import { integer, sqliteTable } from 'drizzle-orm/sqlite-core'

sqliteAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
return {
...schema,
tables: {
...schema.tables,
addedTable: sqliteTable('added_table', {
id: integer('id').primaryKey({ autoIncrement: true }),
}),
},
}
},
],
})
```

One use case is preserving your existing database structure when migrating to Payload. By default, Payload drops the current database schema, which may not be desirable in this scenario.
To quickly generate the Drizzle schema from your database you can use [Drizzle Introspection](https://orm.drizzle.team/kit-docs/commands#introspect--pull)
You should get the `schema.ts` file which may look like this:

```ts
import { sqliteTable, text, uniqueIndex, integer } from 'drizzle-orm/sqlite-core'

export const users = sqliteTable('users', {
id: integer('id').primaryKey({ autoIncrement: true }),
fullName: text('full_name'),
phone: text('phone', {length: 256}),
})

export const countries = sqliteTable(
'countries',
{
id: integer('id').primaryKey({ autoIncrement: true }),
name: text('name', { length: 256 }),
},
(countries) => {
return {
nameIndex: uniqueIndex('name_idx').on(countries.name),
}
},
)

```

You can import them into your config and append to the schema with the `beforeSchemaInit` hook like this:

```ts
import { sqliteAdapter } from '@payloadcms/db-sqlite'
import { users, countries } from '../drizzle/schema'

sqliteAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
return {
...schema,
tables: {
...schema.tables,
users,
countries
},
}
},
],
})
```

Make sure Payload doesn't overlap table names with its collections. For example, if you already have a collection with slug "users", you should either change the slug or `dbName` to change the table name for this collection.


### afterSchemaInit

Runs after the Drizzle schema is built. You can use this hook to modify the schema with features that aren't supported by Payload, or if you want to add a column that you don't want to be in the Payload config.
To extend a table, Payload exposes `extendTable` utillity to the args. You can refer to the [Drizzle documentation](https://orm.drizzle.team/docs/sql-schema-declaration).
The following example adds the `extra_integer_column` column and a composite index on `country` and `city` columns.

```ts
import { sqliteAdapter } from '@payloadcms/db-sqlite'
import { index, integer } from 'drizzle-orm/sqlite-core'
import { buildConfig } from 'payload'

export default buildConfig({
collections: [
{
slug: 'places',
fields: [
{
name: 'country',
type: 'text',
},
{
name: 'city',
type: 'text',
},
],
},
],
db: sqliteAdapter({
afterSchemaInit: [
({ schema, extendTable, adapter }) => {
extendTable({
table: schema.tables.places,
columns: {
extraIntegerColumn: integer('extra_integer_column'),
},
extraConfig: (table) => ({
country_city_composite_index: index('country_city_composite_index').on(
table.country,
table.city,
),
}),
})

return schema
},
],
}),
})

```
2 changes: 2 additions & 0 deletions packages/db-postgres/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ export function postgresAdapter(args: Args): DatabaseAdapterObj<PostgresAdapter>

return createDatabaseAdapter<PostgresAdapter>({
name: 'postgres',
afterSchemaInit: args.afterSchemaInit ?? [],
beforeSchemaInit: args.beforeSchemaInit ?? [],
defaultDrizzleSnapshot,
drizzle: undefined,
enums: {},
Expand Down
15 changes: 15 additions & 0 deletions packages/db-postgres/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,26 @@ import type {
MigrateDownArgs,
MigrateUpArgs,
PostgresDB,
PostgresSchemaHook,
} from '@payloadcms/drizzle/postgres'
import type { DrizzleAdapter } from '@payloadcms/drizzle/types'
import type { DrizzleConfig } from 'drizzle-orm'
import type { PgSchema, PgTableFn, PgTransactionConfig } from 'drizzle-orm/pg-core'
import type { Pool, PoolConfig } from 'pg'

export type Args = {
/**
* Transform the schema after it's built.
* You can use it to customize the schema with features that aren't supported by Payload.
* Examples may include: composite indices, generated columns, vectors
r1tsuu marked this conversation as resolved.
Show resolved Hide resolved
*/
afterSchemaInit?: PostgresSchemaHook[]
/**
* Transform the schema before it's built.
* You can use it to preserve an existing database schema and if there are any collissions Payload will override them.
* To generate Drizzle schema from the database, see [Drizzle Kit introspection](https://orm.drizzle.team/kit-docs/commands#introspect--pull)
r1tsuu marked this conversation as resolved.
Show resolved Hide resolved
*/
beforeSchemaInit?: PostgresSchemaHook[]
idType?: 'serial' | 'uuid'
localesSuffix?: string
logger?: DrizzleConfig['logger']
Expand Down Expand Up @@ -41,6 +54,8 @@ declare module 'payload' {
export interface DatabaseAdapter
extends Omit<Args, 'idType' | 'logger' | 'migrationDir' | 'pool'>,
DrizzleAdapter {
afterSchemaInit: PostgresSchemaHook[]
beforeSchemaInit: PostgresSchemaHook[]
beginTransaction: (options?: PgTransactionConfig) => Promise<null | number | string>
drizzle: PostgresDB
enums: Record<string, GenericEnum>
Expand Down
2 changes: 2 additions & 0 deletions packages/db-sqlite/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ export function sqliteAdapter(args: Args): DatabaseAdapterObj<SQLiteAdapter> {

return createDatabaseAdapter<SQLiteAdapter>({
name: 'sqlite',
afterSchemaInit: args.afterSchemaInit ?? [],
beforeSchemaInit: args.beforeSchemaInit ?? [],
client: undefined,
clientConfig: args.client,
defaultDrizzleSnapshot,
Expand Down
8 changes: 6 additions & 2 deletions packages/db-sqlite/src/init.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import type { DrizzleAdapter } from '@payloadcms/drizzle/types'
import type { Init, SanitizedCollectionConfig } from 'payload'

import { createTableName } from '@payloadcms/drizzle'
import { createTableName, executeSchemaHooks } from '@payloadcms/drizzle'
import { uniqueIndex } from 'drizzle-orm/sqlite-core'
import { buildVersionCollectionFields, buildVersionGlobalFields } from 'payload'
import toSnakeCase from 'to-snake-case'
Expand All @@ -11,8 +11,10 @@ import type { SQLiteAdapter } from './types.js'

import { buildTable } from './schema/build.js'

export const init: Init = function init(this: SQLiteAdapter) {
export const init: Init = async function init(this: SQLiteAdapter) {
let locales: [string, ...string[]] | undefined
await executeSchemaHooks({ type: 'beforeSchemaInit', adapter: this })

if (this.payload.config.localization) {
locales = this.payload.config.localization.locales.map(({ code }) => code) as [
string,
Expand Down Expand Up @@ -132,4 +134,6 @@ export const init: Init = function init(this: SQLiteAdapter) {
})
}
})

await executeSchemaHooks({ type: 'afterSchemaInit', adapter: this })
}
Loading