[DC-3271] Use Jinja template instead of string concatenation when importing from RDR dataset #1691
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dev-michael-schmidt
changed the title
dev-michael-schmidt
force-pushed
the
ms/dc-3271
branch
from
0910aa3 to
a953f63
Compare
dev-michael-schmidt
changed the title
* [DC-3635] Remove portion of query checking primary consent * [DC-3635] Remove fitbit check in CT
…concept domains (#1827) * [DC-3631] Add helpful descriptions * [DC-3631] remove trailing whitespace * [DC-3631] Remove space in function use * [DC-3631] update failure query * [DC-3631] Use meaningful variable name "result" to distinguish and prevent confusion by a single number "1" * [DC-3631] correct variable name in query * [DC-3631] Update and parameterize query * [DC-3631] Update failure logic * [DC-3631] Implement query * [DC-3631] Follow basic Python conventions * [DC-3631] Rename ambiguous "df" to "summary" * [DC-3631] update query summary sections, formatting * [DC-3631] Cleanup staged/testing snippets and update comments * [DC-3631] Add result summary * [DC-3631] Python list does not have a shape * [DC-3631] Remove "pipeline_table" variable * [DC-3631] template table instead of test table procedure_occurrence * [DC-3631] Remove procedure concept into {{table_name}} * [DC-3631] Explicit description on parameter's purpose
…1840) [DC-3658] Updated Participant validation QC check.
…ssifications (#1839) [DC-3659] Updated CT Part2 QC check.
[DC-3650] validation dataset script to validate Hpo instead of cron job.
[DC-3651] Combined Backup script fix.
* [DC-3333] Update venv and docker config * [DC-3333] Update cicleci settings * [DC-3333] Fix failing lint checks * [DC-3333] Revert Pandas for QC notebook compatibility * [DC-3333] Update failing ppi_branching_test * [DC-3333] Update bq to comply with latest version * [DC-3333] Disable custom log temporarily * [DC-3333] Update custom logging * [DC-3333] Delete custom loggers * [DC-3333] Fix the failing unit tests * [DC-3333] Fix the failing unit test * [DC-3333] Fix the failing unit test * [DC-3333] Fix failing load_vocab test * [DC-3333] Remove moz_sql_parser reference * [DC-3333] Update Docker image * [DC-3333] Update gsdk to the latest * [DC-3333] Correct checksum of Google SDK
* [DC-3660] Updated combined QC check. * [DC-3660] Added comment
… aou_death (#1845) * [DC-3673] Delete query * [DC-3673] change "df1" to "result" * [DC-3673] change "df" to "summary" * [DC-3673] Minor formatting
* [DC-3361] Update `fitbit_qc` parameters and descriptions * [DC-3661] suggested changes
* [DC-3668] Ignore race/ethnicity sub categories in ct notebook * [DC-3668] remove unsuppressed concepts from csv
* [DC-3632] Remove five cases of unused imports * [DC-3632] Update 5 more usages of imports, some formatting occurred * [DC-3632] remove 5 more usages of imports, some formatting occurred * [DC-3632] remove 5 more usages of imports, some formatting occurred * [DC-3632] remove final 2 occurrences, some formatting occurred * [DC-3632] Add required import back
dev-michael-schmidt
force-pushed
the
ms/dc-3271
branch
from
59bf610 to
9961da6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
data_steward/tools/import_rdr_dataset.py can be simplified for maintenance and promoting good coding practices by using a jinja template statement. This statement can accept the schema list and any other parameters, allowing the template to cycle through the options using a
{% for %}loop.Scope
Update the loop generating a Python string to use the jinja_env string format/render.
Acceptance Criteria
The same SQL statement is generated with the Jinja template currently developed with the Python-built string.