Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: new migration system #876

Merged
merged 10 commits into from
Apr 27, 2024
Merged

feat: new migration system #876

merged 10 commits into from
Apr 27, 2024

Conversation

fmartingr
Copy link
Member

@fmartingr fmartingr commented Apr 6, 2024

This pull request removes the golang-migrate dependency for a new custom migration system. This new system will leverage existing files but will add the possibility of running Golang code as a migration for portability.

Changes:

  • Removes golang-migrate
  • Adds methods to the DB interface:
    • GetDatabaseSchemaVersion: Retrieves the database schema version
    • SetDatabaseSchemaVersion: Sets the database schema version
    • DBx: returns the underlying dbx.DB object.
  • DB.Migrate now requires a context.Context
  • Moved the database driver imports to their respective package files
  • Simplified all current changes into single migrations (except for MySQL). Added a backwards compatible migration for upgrades to ensure users that haven't run all migrations have the new columns present.

Caveats:

  • Migrations can't go backwards. System can be implemented but in reality this is not likely to happen. The idea is to make migrations backwards compatible (not removing/changing columns). We will see.

@fmartingr fmartingr marked this pull request as ready for review April 8, 2024 10:52
@fmartingr fmartingr requested a review from Monirzadeh April 8, 2024 10:52
@fmartingr fmartingr modified the milestones: 1.7.0, 1.6.3 Apr 8, 2024
Copy link
Collaborator

@Monirzadeh Monirzadeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is fine to me,
thanks

@fmartingr fmartingr merged commit 31767f7 into master Apr 27, 2024
11 checks passed
@fmartingr fmartingr deleted the feat/migrations branch April 27, 2024 05:46
@fmartingr fmartingr modified the milestones: 1.6.3, 1.6.4, 1.8.0, 1.7.0 Apr 27, 2024
Monirzadeh pushed a commit to Monirzadeh/shiori that referenced this pull request Apr 27, 2024
* feat: new migration system

* use newFuncMigration

* database version -> database schema version

* column name

* use path instead of filepath for goembed

* simplified migrations, added backwards compatible migrations
Monirzadeh pushed a commit to Monirzadeh/shiori that referenced this pull request Apr 27, 2024
* feat: new migration system

* use newFuncMigration

* database version -> database schema version

* column name

* use path instead of filepath for goembed

* simplified migrations, added backwards compatible migrations
Monirzadeh pushed a commit to Monirzadeh/shiori that referenced this pull request Apr 27, 2024
* feat: new migration system

* use newFuncMigration

* database version -> database schema version

* column name

* use path instead of filepath for goembed

* simplified migrations, added backwards compatible migrations
truecharts-admin referenced this pull request in truecharts/public Jun 8, 2024
…0@85a47b2 by renovate (#23111)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/go-shiori/shiori](https://togithub.com/go-shiori/shiori) |
minor | `v1.6.3` -> `v1.7.0` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>go-shiori/shiori (ghcr.io/go-shiori/shiori)</summary>

###
[`v1.7.0`](https://togithub.com/go-shiori/shiori/releases/tag/v1.7.0)

[Compare
Source](https://togithub.com/go-shiori/shiori/compare/v1.6.3...v1.7.0)

**Always remember to backup your data before updating.**

#### Notable changes

##### System Theme
([@&#8203;Monirzadeh](https://togithub.com/Monirzadeh))
[#&#8203;924](https://togithub.com/go-shiori/shiori/issues/924)

Shiori now allows you to change the theme to light/dark or follow the
system configuration.

![Screenshot 2024-06-03 at 13 19
39](https://togithub.com/go-shiori/shiori/assets/812088/76e3e062-e36c-4118-84d5-563ad48334cb)

##### New migrations backend
([@&#8203;fmartingr](https://togithub.com/fmartingr))
[#&#8203;876](https://togithub.com/go-shiori/shiori/issues/876)

The underlying migrations system has been rewritten to custom code
removing the [go-migrate
dependency](https://togithub.com/golang-migrate/migrate).

This not only removes one more dependency but also allows for more
control over the migrations process by letting us add run code in a
migration, for example, to update the database schema.

This should be transparent for all users but if you find any problems
please [report
it](https://togithub.com/go-shiori/shiori/issues/new/choose)

##### OpenBSD support
([@&#8203;pacoesteban](https://togithub.com/pacoesteban))
[#&#8203;780](https://togithub.com/go-shiori/shiori/issues/780)

This has been in the works for several months since we broke it around
1.5 but thanks to the above migration changes and some custom database
engine backend for OpenBSD, we are now able to support OpenBSD again. We
added a CI step to get early warnings if we introduce something that
breaks support.

#### What's Changed

- feat: new migration system by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/876](https://togithub.com/go-shiori/shiori/pull/876)
- chore(deps): bump the all group across 1 directory with 4 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/895](https://togithub.com/go-shiori/shiori/pull/895)
- refactor: allow specific sqlite engine for OpenBSD by
[@&#8203;pacoesteban](https://togithub.com/pacoesteban) in
[https://github.com/go-shiori/shiori/pull/780](https://togithub.com/go-shiori/shiori/pull/780)
- chore(deps): bump the all group across 1 directory with 4 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/900](https://togithub.com/go-shiori/shiori/pull/900)
- chore(deps): bump the all group with 4 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/902](https://togithub.com/go-shiori/shiori/pull/902)
- fix: not checking for nil-pointer errors on migrations by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/906](https://togithub.com/go-shiori/shiori/pull/906)
- ci: unify local and ci docker workflows by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/907](https://togithub.com/go-shiori/shiori/pull/907)
- fix: ensure tmp folder is present on docker container by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/910](https://togithub.com/go-shiori/shiori/pull/910)
- deps: update golang dependencies by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/915](https://togithub.com/go-shiori/shiori/pull/915)
- chore(deps): bump the all group with 3 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/908](https://togithub.com/go-shiori/shiori/pull/908)
- chore(deps): bump the all group with 2 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/917](https://togithub.com/go-shiori/shiori/pull/917)
- feat: Home button clear search query by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/916](https://togithub.com/go-shiori/shiori/pull/916)
- chore(deps): bump codecov/codecov-action from 4.4.0 to 4.4.1 in the
all group by [@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/922](https://togithub.com/go-shiori/shiori/pull/922)
- chore: check for avx2 processor feature when trying to run bun by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/920](https://togithub.com/go-shiori/shiori/pull/920)
- ci: fix codecov action by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/928](https://togithub.com/go-shiori/shiori/pull/928)
- fix: incorrect original link in archive page by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/931](https://togithub.com/go-shiori/shiori/pull/931)
- fix : wrong response type for readable endpoint documentation by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/932](https://togithub.com/go-shiori/shiori/pull/932)
- feat: allow selecting light/dark/follow themes in the webui by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/924](https://togithub.com/go-shiori/shiori/pull/924)
- fix: add version to goreleaser archive filename by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/934](https://togithub.com/go-shiori/shiori/pull/934)

#### New Contributors

- [@&#8203;pacoesteban](https://togithub.com/pacoesteban) made their
first contribution in
[https://github.com/go-shiori/shiori/pull/780](https://togithub.com/go-shiori/shiori/pull/780)

**Full Changelog**:
go-shiori/shiori@v1.6.3...v1.7.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zOTkuMyIsInVwZGF0ZWRJblZlciI6IjM3LjM5OS4zIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbImF1dG9tZXJnZSIsInVwZGF0ZS9kb2NrZXIvZ2VuZXJhbC9ub24tbWFqb3IiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants