Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run django db migrations for tests #729

Merged
merged 24 commits into from
Nov 6, 2024

Conversation

michelletran-codecov
Copy link
Contributor

@michelletran-codecov michelletran-codecov commented Sep 19, 2024

This ensures that the table that the models run on are the Django models, rather than the slightly less accurate SQLAlchemy models.

The first 3 commits are related to getting the SQLAlchemy tables created with Django migrations. I've decided to keep the databases for each test separate (because Django transactions tests won't work properly with a persistently open SQLAlchemy DB connection).

Some of the common fixes to the tests:

  • add timezones
  • set created_at timestamp for owner table to be before the patch centric YAML change (the default DB behaviour is to set that time to now, which breaks old legacy tests that check YAML)
  • fix issues with missing not-null fields or unique references

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Copy link

codecov bot commented Sep 19, 2024

Codecov Report

Attention: Patch coverage is 97.94521% with 3 lines in your changes missing coverage. Please review.

Project coverage is 98.01%. Comparing base (1e2c27a) to head (3741275).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
conftest.py 92.30% 3 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #729      +/-   ##
==========================================
- Coverage   98.02%   98.01%   -0.01%     
==========================================
  Files         444      445       +1     
  Lines       36057    36099      +42     
==========================================
+ Hits        35346    35384      +38     
- Misses        711      715       +4     
Flag Coverage Δ
integration 98.01% <97.94%> (-0.01%) ⬇️
unit 98.01% <97.94%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.01% <100.00%> (+0.02%) ⬆️
OutsideTasks 98.05% <96.87%> (-0.01%) ⬇️
Files with missing lines Coverage Δ
database/tests/factories/core.py 99.39% <100.00%> (ø)
database/tests/unit/test_model_utils.py 100.00% <100.00%> (ø)
django_scaffold/tests_settings.py 100.00% <100.00%> (ø)
helpers/tests/unit/test_github_installation.py 100.00% <100.00%> (ø)
..._scripts/tests/test_backfill_daily_test_rollups.py 100.00% <100.00%> (ø)
...f_scripts/tests/test_backfill_test_flag_bridges.py 100.00% <100.00%> (ø)
one_off_scripts/tests/utils.py 100.00% <100.00%> (ø)
...vices/bundle_analysis/notify/tests/test_helpers.py 100.00% <ø> (ø)
services/notification/notifiers/tests/conftest.py 100.00% <100.00%> (ø)
...cation/notifiers/tests/integration/test_comment.py 92.78% <100.00%> (ø)
... and 17 more

... and 2 files with indirect coverage changes

@codecov-notifications
Copy link

codecov-notifications bot commented Sep 19, 2024

Codecov Report

Attention: Patch coverage is 97.94521% with 3 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
conftest.py 92.30% 3 Missing ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #729      +/-   ##
==========================================
- Coverage   98.02%   98.01%   -0.01%     
==========================================
  Files         444      445       +1     
  Lines       36057    36099      +42     
==========================================
+ Hits        35346    35384      +38     
- Misses        711      715       +4     
Flag Coverage Δ
integration 98.01% <97.94%> (-0.01%) ⬇️
unit 98.01% <97.94%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.01% <100.00%> (+0.02%) ⬆️
OutsideTasks 98.05% <96.87%> (-0.01%) ⬇️
Files with missing lines Coverage Δ
database/tests/factories/core.py 99.39% <100.00%> (ø)
database/tests/unit/test_model_utils.py 100.00% <100.00%> (ø)
django_scaffold/tests_settings.py 100.00% <100.00%> (ø)
helpers/tests/unit/test_github_installation.py 100.00% <100.00%> (ø)
..._scripts/tests/test_backfill_daily_test_rollups.py 100.00% <100.00%> (ø)
...f_scripts/tests/test_backfill_test_flag_bridges.py 100.00% <100.00%> (ø)
one_off_scripts/tests/utils.py 100.00% <100.00%> (ø)
...vices/bundle_analysis/notify/tests/test_helpers.py 100.00% <ø> (ø)
services/notification/notifiers/tests/conftest.py 100.00% <100.00%> (ø)
...cation/notifiers/tests/integration/test_comment.py 92.78% <100.00%> (ø)
... and 17 more

... and 2 files with indirect coverage changes

@codecov-qa
Copy link

codecov-qa bot commented Sep 19, 2024

Codecov Report

Attention: Patch coverage is 97.94521% with 3 lines in your changes missing coverage. Please review.

Project coverage is 98.01%. Comparing base (1e2c27a) to head (3741275).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
conftest.py 92.30% 3 Missing ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #729      +/-   ##
==========================================
- Coverage   98.02%   98.01%   -0.01%     
==========================================
  Files         444      445       +1     
  Lines       36057    36099      +42     
==========================================
+ Hits        35346    35384      +38     
- Misses        711      715       +4     
Flag Coverage Δ
integration 98.01% <97.94%> (-0.01%) ⬇️
unit 98.01% <97.94%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.01% <100.00%> (+0.02%) ⬆️
OutsideTasks 98.05% <96.87%> (-0.01%) ⬇️
Files with missing lines Coverage Δ
database/tests/factories/core.py 99.39% <100.00%> (ø)
database/tests/unit/test_model_utils.py 100.00% <100.00%> (ø)
django_scaffold/tests_settings.py 100.00% <100.00%> (ø)
helpers/tests/unit/test_github_installation.py 100.00% <100.00%> (ø)
..._scripts/tests/test_backfill_daily_test_rollups.py 100.00% <100.00%> (ø)
...f_scripts/tests/test_backfill_test_flag_bridges.py 100.00% <100.00%> (ø)
one_off_scripts/tests/utils.py 100.00% <100.00%> (ø)
...vices/bundle_analysis/notify/tests/test_helpers.py 100.00% <ø> (ø)
services/notification/notifiers/tests/conftest.py 100.00% <100.00%> (ø)
...cation/notifiers/tests/integration/test_comment.py 92.78% <100.00%> (ø)
... and 17 more

... and 2 files with indirect coverage changes

Copy link

codecov-public-qa bot commented Sep 19, 2024

Codecov Report

Attention: Patch coverage is 97.94521% with 3 lines in your changes missing coverage. Please review.

Project coverage is 98.01%. Comparing base (1e2c27a) to head (3741275).

✅ All tests successful. No failed tests found.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #729      +/-   ##
==========================================
- Coverage   98.02%   98.01%   -0.01%     
==========================================
  Files         444      445       +1     
  Lines       36057    36099      +42     
==========================================
+ Hits        35346    35384      +38     
- Misses        711      715       +4     
Flag Coverage Δ
integration 98.01% <97.94%> (-0.01%) ⬇️
unit 98.01% <97.94%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 96.01% <100.00%> (+0.02%) ⬆️
OutsideTasks 98.05% <96.87%> (-0.01%) ⬇️
Files Coverage Δ
database/tests/factories/core.py 99.39% <100.00%> (ø)
database/tests/unit/test_model_utils.py 100.00% <100.00%> (ø)
django_scaffold/tests_settings.py 100.00% <100.00%> (ø)
helpers/tests/unit/test_github_installation.py 100.00% <100.00%> (ø)
..._scripts/tests/test_backfill_daily_test_rollups.py 100.00% <100.00%> (ø)
...f_scripts/tests/test_backfill_test_flag_bridges.py 100.00% <100.00%> (ø)
one_off_scripts/tests/utils.py 100.00% <100.00%> (ø)
...vices/bundle_analysis/notify/tests/test_helpers.py 100.00% <ø> (ø)
services/notification/notifiers/tests/conftest.py 100.00% <100.00%> (ø)
...cation/notifiers/tests/integration/test_comment.py 92.78% <100.00%> (ø)
... and 17 more

... and 2 files with indirect coverage changes

Copy link

github-actions bot commented Oct 31, 2024

This PR includes changes to shared. Please review them here: codecov/shared@6e68957...83b3376

@michelletran-codecov michelletran-codecov force-pushed the run_django_db_migrations_for_tests branch 6 times, most recently from 9645839 to b00b977 Compare November 2, 2024 00:04
@michelletran-codecov michelletran-codecov force-pushed the run_django_db_migrations_for_tests branch from 0be321f to d60b457 Compare November 5, 2024 19:51
This was originally bootstrapping separately with Django and SQLAlchemy
creating separate databases. This ensures that Django bootstraps the test
database with (more canonical) Django models. SQLAlchemy will reuse the
database that Django bootstraps for testing purposes.
There are quite a few tables that are not moved to `shared` yet. So for the sake
of getting tests to work, I'm bootstrapping them with the SQLAlchemy models.
This is because Django's TransactionTest do a TRUNCATE flush at the end to clean up
the database. However, this fails due to SQLAlchemy's ongoing transactions in the DB.
So, it's probably best to keep the 2 types of tests separate for now.

Refactor SQLAlchemy and Django instantiation
This is because the actual DB value is timezone aware.
* null constraints on installation_ids
* default yaml for owners created after a certain date
The owner.createstamp is now defaulting to now, which breaks all the tests
that assumes that we are before the patch centric YAML selection. Now force the
createstamp of the owner
These tests are reusing ids, which is causing some problems with conflicting keys.
I just used different ids, but we might want something more systemic to generate
unique ids everytime we create a new pull object in the future.
"completed" is not a valid commit state. This was supposed to be "complete".
The check was hard-coding the ids, but that's particularly fragile.
This is more stable and less flaky or prone to duplicate key errors.
@michelletran-codecov michelletran-codecov force-pushed the run_django_db_migrations_for_tests branch from ef8eaf6 to 3741275 Compare November 6, 2024 14:09
@michelletran-codecov michelletran-codecov added this pull request to the merge queue Nov 6, 2024
Merged via the queue into main with commit acca45b Nov 6, 2024
22 of 27 checks passed
@michelletran-codecov michelletran-codecov deleted the run_django_db_migrations_for_tests branch November 6, 2024 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants