Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added index for redirects lookup #21783

Merged
merged 5 commits into from
Dec 3, 2024
Merged

Added index for redirects lookup #21783

merged 5 commits into from
Dec 3, 2024

Conversation

9larsons
Copy link
Contributor

@9larsons 9larsons commented Dec 2, 2024

ref https://linear.app/ghost/issue/ENG-1811/

  • added first migration for truncating column length; required in order to be able to add an index
  • added second migration for creating the index

After sending a newsletter, Ghost can struggle with the amount of incoming traffic (particularly link scanners) with particularly large pools of recipients. Part of this is link lookup, which an index has shown to help with when added manually on the db. Given that, this change is to make it native.

I ran a stats service query to gather data on usage, as well as did code review, and I've not seen any use of this table that isn't using the 8 digit slug, i.e. /r/12345678, or 11 characters total. I made this 191 in case that use case changes, while still allowing us the value of an index.

Note: SQLite does not support altering columns to my knowledge, so we have to create a temp table with the new column data to push back in. This should be ok because use of SQLite should be limited. In the past, I've seen migrations skipped for SQLite and I'd rather not do that here if able.

ref https://linear.app/ghost/issue/ENG-1811/
- added first migration for truncating column length; required in order to be able to add an index
- added second migration for creating the index

After sending a newsletter, Ghost can struggle with the amount of incoming traffic (particularly link scanners) with particularly large pools of recipients. Part of this is link lookup, which an index has shown to help with when added manually on the db. Given that, this change is to make it native.

I ran a stats service query to gather data on usage, as well as did code review, and I've not seen any use of this table that isn't using the 8 digit slug, i.e. /r/12345678, or 11 characters total. I made this 191 in case that use case changes, while still allowing us the value of an index.
@github-actions github-actions bot added the migration [pull request] Includes migration for review label Dec 2, 2024
Copy link
Contributor

github-actions bot commented Dec 2, 2024

It looks like this PR contains a migration 👀
Here's the checklist for reviewing migrations:

General requirements

  • Satisfies idempotency requirement (both up() and down())
  • Does not reference models
  • Filename is in the correct format (and correctly ordered)
  • Targets the next minor version
  • All code paths have appropriate log messages
  • Uses the correct utils
  • Contains a minimal changeset
  • Does not mix DDL/DML operations
  • Tested in MySQL and SQLite

Schema changes

  • Both schema change and related migration have been implemented
  • For index changes: has been performance tested for large tables
  • For new tables/columns: fields use the appropriate predefined field lengths
  • For new tables/columns: field names follow the appropriate conventions
  • Does not drop a non-alpha table outside of a major version

Data changes

  • Mass updates/inserts are batched appropriately
  • Does not loop over large tables/datasets
  • Defends against missing or invalid data
  • For settings updates: follows the appropriate guidelines

@9larsons
Copy link
Contributor Author

9larsons commented Dec 2, 2024

Note, knex has a limit on the character allowed (191), and MySQL has some limits as well.

Copy link
Contributor

@allouis allouis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the comment by @daniellockyer this looks good. If we can rely on knex to handle the table rename dance for us then we should!

@9larsons 9larsons enabled auto-merge (squash) December 3, 2024 13:54
@9larsons 9larsons merged commit 3e0864a into main Dec 3, 2024
21 checks passed
@9larsons 9larsons deleted the add-index-redirects branch December 3, 2024 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
migration [pull request] Includes migration for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants