Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(changelog): should exclude file contents, search indexes and mg_ fields to keep storage use down #4305

Open
wants to merge 26 commits into
base: master
Choose a base branch
from

Conversation

mswertz
Copy link
Member

@mswertz mswertz commented Oct 2, 2024

What are the main changes you did:

  • changelog should skip contents (for file contents), text search index, and the mg fields
  • change the PR so it uses file type to skip (suggest to generate the fields to be included at update time instead on each execution)
  • only store the 'changed columns' in old (which are all columns in delete)

how to test:

  • existing unit tests

notes:

  • by removing file content we disable a future 'undo' for image columns

@mswertz mswertz changed the title Fix/change log skip file contents fix(sql): changelog should exclude file contents and search indexes Oct 2, 2024
@mswertz mswertz marked this pull request as draft October 2, 2024 20:56
@mswertz mswertz changed the title fix(sql): changelog should exclude file contents and search indexes fix(changelog): should exclude file contents, search indexes and mg_ fields to keep storage use down Oct 2, 2024
@mswertz mswertz marked this pull request as ready for review October 2, 2024 22:13
@mswertz mswertz marked this pull request as draft October 8, 2024 06:46
@mswertz mswertz marked this pull request as ready for review October 8, 2024 21:00
}

public static String buildAuditTrigger(String schemaName, String tableName) {
return """
CREATE TRIGGER %3$s_audit
CREATE OR REPLACE TRIGGER %3$s_audit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these should not be a trigger when the trigger is add , should fail fast

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

???

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the code assumes there is no trigger with this name , if there is a trigger with this name, something has gone (very) wrong and we should know , not suppress the error

@@ -220,7 +236,7 @@ static void executeMigrationFile(Database db, String sqlFile, String message) {
jooq.settings().setQueryTimeout(THREE_MINUTES);
String sql = new String(Migrations.class.getResourceAsStream(sqlFile).readAllBytes());
jooq.execute(sql);
logger.debug(message + "(file = " + sqlFile);
logger.info(message + "(file = " + sqlFile);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

info ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently we cannot see that migrations are executed. I think this is important enough to be 'info'

Copy link
Contributor

@connoratrug connoratrug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • should be marked as feature instead of fix as it changes behaviour ( and even the log )
  • not sure if we need the feature anymore

@mswertz
Copy link
Member Author

mswertz commented Oct 16, 2024

  • not sure if we need the feature anymore

I think we do. File contents are disturbing the changelog viewing and take much space. If we would want to log those it should be elsewhere.

Copy link

sonarcloud bot commented Oct 16, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants