Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto initialize algorithms in DB #89

Merged
merged 59 commits into from
Oct 21, 2024
Merged

Auto initialize algorithms in DB #89

merged 59 commits into from
Oct 21, 2024

Conversation

ericbuckley
Copy link
Collaborator

@ericbuckley ericbuckley commented Oct 18, 2024

Description

Configuring the code to optionally load in initial algorithms after the tables have been initialized. Doing so, removes our needs now for having to use a database migration tool for the near future.

Related Issues

closes #57

Additional Notes

  • This is contentious, but have removed alembic. Using alembic has required a lot of manual effort to get migrations correct, with little benefit this early in a project. Our schema hasn't stabilized yet, and we don't have any systems running the code in production, so it's of little value. Over time, we may need to bring this back to safely adjust production schemas, but now is not the time.
  • Created assets/initial_algorithms.json with the two existing DIBBs algorithms. We can use this for our test cases, and also initializing a new database with the data, if the INITIAL_ALGORITHMS env variable is set. This also gives customers another option for managing their algorithms, in they can write their own and set the INITIAL_ALGORITHMS env variable to load that json file.
  • Capitalized schemas.pii.Features. The blocking keys were capitalized, so making them both capitalized will hopefully reduce errors in writing the algorithm specs.
  • Consolidated some of the assets and testing assets, as there were duplicates and there was some testing files in the top level one.

<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->

Checklist

Please review and complete the following checklist before submitting your pull request:

  • I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
  • I have reviewed my changes to ensure they are clear, concise, and well-documented.
  • I have updated the documentation, if applicable.
  • I have added or updated test cases to cover my changes, if applicable.
  • I have minimized the number of reviewers to include only those essential for the review.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

  • The code follows best practices and conventions.
  • The changes implement the desired functionality or fix the reported issue.
  • The tests cover the new changes and pass successfully.
  • Any potential edge cases or error scenarios have been considered.

@ericbuckley ericbuckley added the feature New feature or request label Oct 18, 2024
@ericbuckley ericbuckley self-assigned this Oct 18, 2024
Copy link

codecov bot commented Oct 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.51%. Comparing base (bc0d93f) to head (c0c55ea).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #89      +/-   ##
==========================================
+ Coverage   93.26%   93.51%   +0.24%     
==========================================
  Files          19       19              
  Lines         936      971      +35     
==========================================
+ Hits          873      908      +35     
  Misses         63       63              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ericbuckley ericbuckley marked this pull request as ready for review October 18, 2024 18:30
Copy link
Contributor

@alhayward alhayward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Left a few quick questions & 1 suggestion. The justification for moving away from Alembic for now makes sense.

assets/initial_algorithms.json Outdated Show resolved Hide resolved
assets/initial_algorithms.json Show resolved Hide resolved
src/recordlinker/linking/link.py Show resolved Hide resolved
src/recordlinker/main.py Show resolved Hide resolved
tests/unit/linking/test_link.py Show resolved Hide resolved
update to the basic algorithm description

Co-authored-by: Alex Hayward <[email protected]>
Copy link
Contributor

@alhayward alhayward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ericbuckley ericbuckley merged commit dbe3761 into main Oct 21, 2024
7 checks passed
@ericbuckley ericbuckley deleted the feature/57-auto-init-db branch October 21, 2024 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Auto initialize database with algorithms
3 participants