Demo rfc80 before #11048

alisman · 2024-10-06T06:07:02Z

Fix # (see https://help.github.com/en/articles/closing-issues-using-keywords)

Describe changes proposed in this pull request:

a
b

Checks

The commit log is comprehensible. It follows 7 rules of great commit messages. We can fix this during merge by using a squash+merge if necessary
Has tests or has a separate issue that describes the types of test that should be created. If no test is included it should explicitly be mentioned in the PR why there is no test.
Is this PR adding logic based on one or more clinical attributes? If yes, please make sure validation for this attribute is also present in the data validation / data loading layers (in backend repo) and documented in File-Formats Clinical data section!
Make sure your PR has one of the labels defined in https://github.com/cBioPortal/cbioportal/blob/master/.github/release-drafter.yml

Any screenshots or GIFs?

If this is a new visual feature please add a before/after screenshot or gif
here with e.g. Giphy CAPTURE or Peek

Notify reviewers

Read our Pull request merging
policy. It can help to figure out who worked on the
file before you. Please use git blame <filename> to determine that
and notify them either through slack or by assigning them as a reviewer on the PR

* Add Columnar SQL file to init Clickhouse DB * Refactored Mapper xml to extract StudyViewFilterMapper

* ✅ Add Unit test for StudyViewMapper Clickhouse * ✅ Update db props to include mysql and clickhouse datasources to fix tests * Address comments * Rename package to clickhouse * Update to static final * Use bean name instead of qualifier

* Create new wide table sql file and rename package * Remove genomic_event view * Add AlterationFilter to mutated_genes endpoint * Add AlterationFilter to mutated-genes endpoint * Fix unit test * Fix sonar issues * Add test for mutation types and status * remove unused imports

* add missing poc clinical data binning function

* Add sample_mv materialized view and use it in mappers

* Add Support for TotalProfiledCase Counts for Mutated-genes endpoint. * Create sql files to create new tables * Add unit test for totalProfiledCount * Add matching gene panel ids * Add TotalProfiledCountsWithoutPanelData * Add profileCount for genes without gene panel data * Add Comments for SQL * Update matching Gene Panel Ids * Clean up code * Fix test * Add query to get correct Gene Panels * Fix unit test * Add comments

* working poc * refactor logic into service, so clean * refactor for parameters builder, simplify min max logic, streamline service call * remove unused services and imports * remove more unused imports

* Implement molecular profile count endpoint using Clickhouse * Cleanup

* ✨ Add CNA Gene Endpoint * 🐛 Fix StudyViewFilterMapper.xml to allow ability to filter on gene and alteration * Fix merge conflict * Address comments * Fix unit tests * Fix sonar issues

* ✨ Add StructuralVariant-genes endpoint * Fix sonar issues * Update MatchingGenePanel request to return list * Create and use sample_derive * Update where sample_derived is stored to fix unit test

Co-authored-by: Bryan Lai <[email protected]>

* use clinical_data_derived instead of sample_clinical_attribute_numeric_mv and patient_clinical_attribute_numeric_mv * use clinical_attribute_meta instead of sample_clinical_attribute_numeric_mv and patient_clinical_attribute_numeric_mv * remove unused clinical data count methods and SQL * fix numericalClinicalDataCountFilter * Move CategoricalClinicalAttributeFilter to repository * remove unused columns * Add override to methods --------- Co-authored-by: haynescd <[email protected]>

…0857) * Add patient_id column to genomic_event_derived * Update sql to convert list of patients to list of samples

* refactor to use clickhouse * filter out empty attr values * edit comment * fix sonarcloud issues * use parallel stream, shaves off 5s * use newer mapping annotation

* simplify clinical data binning related SQL * fix numericalClinicalDataCountFilter

…ase64) (#10898)

* Add Clinical Events Timeline counts endpoint * Add comments for StudyViewFilter

* Fix alteration count services * Only add genes that are protein coding to gene_panel_to_gene table for WES * fix test * fix sonar issues * Add documentation for filtering out WES

…ent) (#10919)

* Add Sample Count Report Endpoint * Push counting logic to sql * Remove unused methods * Update PatientTreatments to pass int from db for totalPatientTreatmentCount * Fix tests * Add unit test for sample Treatments * Remove unused import * Fix indentation

* Fix Sample Treatment Endpoint * Fix Sample ID only StudyView Filter

* add unit tests for molecular-profile-counts-service * fix multiple-study scenario Update method name getGenomicDataCounts -> getMolecularProfileSampleCounts

Clickhouse implementations of mutation data counts & genomic data counts services --------- Co-authored-by: Qi-Xuan Lu <[email protected]>

* Add tests for case list sample counts endpoint and fix merge of lists across studies

* Implement custom-data-bin-counts associated filter * Create StudyViewFilterContext and Helper --------- Co-authored-by: Bryan Lai <[email protected]> Co-authored-by: Charles Haynes <[email protected]>

#10959) * Update Treatment Endpoint to handle event_type TREATMENT, previously only searched on Treatment * Handle all variations of Treatment

…#10974)

* Genomic and Generic Assay bin counts endpoints

fix clinical data NA counts for multiple studies --------- Co-authored-by: lismana <[email protected]>

* Fix CNA table definition

Fix rfc80 build

…10986) * 🐛 Fix Mutated Genes Filter bug with Or logic * ✅ Add Tests for Mutated-genes,Cna-genes, and structural-variant filters

haynescd and others added 30 commits April 8, 2024 12:35

➕ Add infra to support multi db and Columnar DataSource Properties

db9e6ea

Add Columnar support for Mybatis

7d8b9f7

Add Columnar Study View Controller /api/column-store

8b07d3f

Feature/ch/update poc base branch (#10776)

63dde5f

* Add Columnar SQL file to init Clickhouse DB * Refactored Mapper xml to extract StudyViewFilterMapper

use regular views instead of materialized views (#10793)

77d07e0

Add missing POC clinical data binning function (#10778)

d24648a

* add missing poc clinical data binning function

Create sample materialized view and use it (#10822)

e37fa86

* Add sample_mv materialized view and use it in mappers

RFC80:density plot endpoint(#10837)

1d1606b

* working poc * refactor logic into service, so clean * refactor for parameters builder, simplify min max logic, streamline service call * remove unused services and imports * remove more unused imports

Fix clinical data counts query/filter using new derived table (#10833)

464f442

Molecular-profile-counts service implementation (#10840)

6998fc0

* Implement molecular profile count endpoint using Clickhouse * Cleanup

Fix clinical data scatterplot (point at new derived cd table) (#10846)

0b8c89e

Rfc80/add cna genes endpoint (#10845)

bb5456a

* ✨ Add CNA Gene Endpoint * 🐛 Fix StudyViewFilterMapper.xml to allow ability to filter on gene and alteration * Fix merge conflict * Address comments * Fix unit tests * Fix sonar issues

✨ Add StructuralVariant-genes endpoint (#10854)

505847b

* ✨ Add StructuralVariant-genes endpoint * Fix sonar issues * Update MatchingGenePanel request to return list * Create and use sample_derive * Update where sample_derived is stored to fix unit test

Implement study view sample-lists-counts endpoint in Clickhouse (#10849)

74b735d

Co-authored-by: Bryan Lai <[email protected]>

Fix class import (#10856)

110236b

Rfc80/fix patient level filtering for alteration counts endpoints (#1…

14b7105

…0857) * Add patient_id column to genomic_event_derived * Update sql to convert list of patients to list of samples

🐛 Fix totalCount calculation when WES not available (#10858)

2b87a26

Consolidate SQL files used to create CH Tables (#10867)

79d36e7

Demo rfc80 poc violin plot (#10863)

2e02f21

* refactor to use clickhouse * filter out empty attr values * edit comment * fix sonarcloud issues * use parallel stream, shaves off 5s * use newer mapping annotation

replace sample_mv references with sample_derived (#10880)

0fc686b

🔧 Update Sonar Cloud Token (#10888)

f602587

Simplify clinical data binning related SQL (#10823)

0dfb346

* simplify clinical data binning related SQL * fix numericalClinicalDataCountFilter

Fix patient_derived table population (patient stable id was getting b…

6df137f

…ase64) (#10898)

Always load frontend over https (localdev mode) (#10901)

3455f97

Rfc80/clinical event timeline endpoint (#10899)

77bf77f

* Add Clinical Events Timeline counts endpoint * Add comments for StudyViewFilter

Fix alteration count services (#10911)

18a5ee7

* Fix alteration count services * Only add genes that are protein coding to gene_panel_to_gene table for WES * fix test * fix sonar issues * Add documentation for filtering out WES

alisman and others added 28 commits August 1, 2024 21:01

Hide replacement clickhouse endpoints from swagger (except new treatm…

e935c41

…ent) (#10919)

Rfc80/update open api annotation for sampleTreatment Endpoint (#10929)

d62c6e5

* Fix Sample Treatment Endpoint * Fix Sample ID only StudyView Filter

Molecular-profile-counts-service (#10934)

280dd58

* add unit tests for molecular-profile-counts-service * fix multiple-study scenario Update method name getGenomicDataCounts -> getMolecularProfileSampleCounts

Add entrez_id to alteration Count endpoints (#10936)

d7af5e5

Demo rfc80 poc mutation data counts & genomic data counts (#10807)

956fb33

Clickhouse implementations of mutation data counts & genomic data counts services --------- Co-authored-by: Qi-Xuan Lu <[email protected]>

Add tests for case list sample counts endpoint (#10935)

b03b615

* Add tests for case list sample counts endpoint and fix merge of lists across studies

add unit tests for clinical data counts sql (#10933)

776fbae

Implement custom-data-bin-counts endpoint (#10827)

05de8ff

* Implement custom-data-bin-counts associated filter * Create StudyViewFilterContext and Helper --------- Co-authored-by: Bryan Lai <[email protected]> Co-authored-by: Charles Haynes <[email protected]>

normalize clinical data (#10943)

1872db1

improve numerical clinical data filter (#10951)

f456bd5

Update Treatment Endpoint to handle event_type TREATMENT, previously … (

b36bffb

#10959) * Update Treatment Endpoint to handle event_type TREATMENT, previously only searched on Treatment * Handle all variations of Treatment

Remove need to filter via Patient-IDs, All filters utilize sample-ids (…

ac3cb9f

…#10974)

add api test specs to backend (#10979)

5db0254

fix api test merge writer (#10980)

e2d502a

Genomic and Generic Assay bin counts endpoints (#10964)

10c4a87

* Genomic and Generic Assay bin counts endpoints

Fix clinical data NA counts for multiple studies (#10978)

59d84c9

fix clinical data NA counts for multiple studies --------- Co-authored-by: lismana <[email protected]>

Demo rfc80 poc mutation data count unit tests (#10981)

092773e

* Fix CNA table definition

Delete custom specs (#10987)

33df3db

Fix GenomicDataFilterTest

a415fa2

Merge pull request #10988 from dippindots/fix-rfc80-build

a2ba293

Fix rfc80 build

Rfc80/add tests mutated-genes,cna-genes,structural-variants filters (#…

a4ff971

…10986) * 🐛 Fix Mutated Genes Filter bug with Or logic * ✅ Add Tests for Mutated-genes,Cna-genes, and structural-variant filters

Set custom http header with elapsed time on all api endpoints (#10994)

a475fc9

fix clinical data NA count logic (#10991)

21a0062

Remove Sample Treatments with no counts (#10995)

8c2425d

All custom header elapsed-time on CORS headers (#10999)

27327de

Remove caching from cd-bin-counts (#11001)

37c79cc

add missing study view filters for clinical data binning (#11007)

b5b2c02

alisman added the do not merge label Oct 6, 2024

alisman closed this Oct 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Demo rfc80 before #11048

Demo rfc80 before #11048

alisman commented Oct 6, 2024

Demo rfc80 before #11048

Demo rfc80 before #11048

Conversation

alisman commented Oct 6, 2024

Checks

Any screenshots or GIFs?

Notify reviewers