Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: malariagen/malariagen-data-python
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v13.0.1
Choose a base ref
...
head repository: malariagen/malariagen-data-python
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Mar 26, 2024

  1. add plink converyt function, faff around with some tests

    Tristan Dennis authored and Tristan Dennis committed Mar 26, 2024
    Copy the full SHA
    ea54da7 View commit details

Commits on Apr 11, 2024

  1. temp fix for avoiding error in mapping all-allele -> biallelic allele…

    …s, avoids using dask
    Tristan Dennis authored and Tristan Dennis committed Apr 11, 2024
    Copy the full SHA
    6afd608 View commit details
  2. updated plink converter to use biallelic_snp_calls with tmp. amended …

    …variant_allele mapping
    Tristan Dennis authored and Tristan Dennis committed Apr 11, 2024
    Copy the full SHA
    3268640 View commit details

Commits on Jun 21, 2024

  1. Drops a few unused columns

    jonbrenas committed Jun 21, 2024
    Copy the full SHA
    df13732 View commit details

Commits on Aug 2, 2024

  1. Not sure what is going on

    jonbrenas committed Aug 2, 2024
    Copy the full SHA
    ab45528 View commit details

Commits on Aug 8, 2024

  1. Copy the full SHA
    ab6f28c View commit details
  2. Added chunking of allele_mapping array so that da.map blocks works

    Removed numba decorator for the apply_allele_mapping function, for now
    Tristan Dennis authored and Tristan Dennis committed Aug 8, 2024
    Copy the full SHA
    5e502bf View commit details
  3. numba works fine

    Tristan Dennis authored and Tristan Dennis committed Aug 8, 2024
    Copy the full SHA
    31f6469 View commit details

Commits on Aug 9, 2024

  1. clear up notebook

    Tristan Dennis authored and Tristan Dennis committed Aug 9, 2024
    Copy the full SHA
    4ef5e9c View commit details

Commits on Aug 14, 2024

  1. Copy the full SHA
    3a3ab7b View commit details

Commits on Aug 20, 2024

  1. Merge branch 'master' into plink-converter-2024-03-26

    jonbrenas authored Aug 20, 2024
    Copy the full SHA
    98767fc View commit details

Commits on Aug 27, 2024

  1. remove dud notebook, tidy up to_plink.py

    Tristan Dennis authored and Tristan Dennis committed Aug 27, 2024
    Copy the full SHA
    af47433 View commit details

Commits on Aug 29, 2024

  1. Used the same kwargs method as in the CNV function

    jonbrenas committed Aug 29, 2024
    Copy the full SHA
    699c1ab View commit details

Commits on Sep 2, 2024

  1. Copy the full SHA
    0570a1b View commit details

Commits on Sep 5, 2024

  1. Copy the full SHA
    65fce2c View commit details

Commits on Sep 16, 2024

  1. Merge branch 'master' into 583-GWSS-colors

    leehart authored Sep 16, 2024
    Copy the full SHA
    bfabef4 View commit details
  2. Update switcher.json

    Removed some minor version options due to long list causing UI issues.
    leehart authored Sep 16, 2024
    Copy the full SHA
    75c9872 View commit details
  3. Update switcher.json

    leehart authored Sep 16, 2024
    Copy the full SHA
    9a62297 View commit details
  4. Upgrade scikit-allel to resolve issues running large PCAs on a distri…

    …buted cluster (#603)
    
    * upgrade scikit-allel
    
    * update lock file
    
    * fix broken poetry lock
    alimanfoo authored Sep 16, 2024
    Copy the full SHA
    7595269 View commit details
  5. Merge branch 'master' into update_switcher_2024-09-16

    leehart authored Sep 16, 2024
    Copy the full SHA
    719f589 View commit details

Commits on Sep 17, 2024

  1. Upgrade scikit-allel minimum version 1.3.13 to resolve issues with ru…

    …nning large PCAs on a distributed cluster (#605)
    
    * upgrade scikit-allel minimum version
    
    * poetry update
    alimanfoo authored Sep 17, 2024
    Copy the full SHA
    da6b7bb View commit details
  2. Merge branch 'master' into update_switcher_2024-09-16

    leehart authored Sep 17, 2024
    Copy the full SHA
    0c6db27 View commit details
  3. Copy the full SHA
    ca8624b View commit details

Commits on Sep 19, 2024

  1. add plink converter test

    Tristan Dennis authored and Tristan Dennis committed Sep 19, 2024
    Copy the full SHA
    96eb157 View commit details
  2. update plink converter test to include test for created files

    Tristan Dennis authored and Tristan Dennis committed Sep 19, 2024
    Copy the full SHA
    98700e3 View commit details
  3. tidy, add tests, refactor snp_data call

    Tristan Dennis authored and Tristan Dennis committed Sep 19, 2024
    Copy the full SHA
    6db9db8 View commit details
  4. Merge branch 'master' into 583-GWSS-colors

    leehart authored Sep 19, 2024
    Copy the full SHA
    2e8b4c4 View commit details
  5. Merge pull request #602 from malariagen/update_switcher_2024-09-16

    Update switcher.json
    leehart authored Sep 19, 2024
    Copy the full SHA
    dbbe02b View commit details
  6. Merge branch 'master' into 583-GWSS-colors

    leehart authored Sep 19, 2024
    Copy the full SHA
    8db7a1c View commit details

Commits on Sep 20, 2024

  1. pass through chunks (#613)

    alimanfoo authored Sep 20, 2024
    Copy the full SHA
    68ff103 View commit details
  2. Handle dask distributed progress (#614)

    * distributed progress
    
    * test local cluster
    alimanfoo authored Sep 20, 2024
    Copy the full SHA
    628f198 View commit details
  3. More options for automatic sizing of chunks (#615)

    * ndauto
    
    * typing
    
    * typing
    
    * fix typing
    
    * check it works
    
    * dunno
    alimanfoo authored Sep 20, 2024
    Copy the full SHA
    4eccc6f View commit details
  4. Fix typos in docs (#610)

    * Fix typo in Af1.rst
    
    * Fix typo in Ag3.rst
    
    * Fix typo in Amin1.rst
    leehart authored Sep 20, 2024
    Copy the full SHA
    a64cebe View commit details
  5. Add parameters to the Anopheles pca function to allow better handling…

    … of outliers (#616)
    
    * add exclude_samples parameter to pca
    
    * add fit_exclude_samples parameter to pca
    
    * refine tests
    alimanfoo authored Sep 20, 2024
    Copy the full SHA
    ce614ae View commit details
  6. support fractions for max_missing_an and min_minor_ac parameters (#617)

    alimanfoo authored Sep 20, 2024
    Copy the full SHA
    5ec5de3 View commit details
  7. Tried to avoid the risk that we ask randint to find an imaginary number

    jonbrenas committed Sep 20, 2024
    Copy the full SHA
    4b49866 View commit details
  8. Merge branch 'master' into 618-bad-random-value

    jonbrenas authored Sep 20, 2024
    Copy the full SHA
    b8cd642 View commit details

Commits on Sep 22, 2024

  1. Handle chunks parameter given as a size in memory; improve default ch…

    …unks (#622)
    
    * Update util.py
    
    Add option for special handling of chunks parameter given as a size in memory (number of bytes).
    
    * Update base_params.py
    
    * refactor and pass through chunks
    
    * fix urls in notebooks
    
    * fix bucket names
    
    * fix directory names
    
    * add missing directories
    alimanfoo authored Sep 22, 2024
    Copy the full SHA
    57c6d8a View commit details

Commits on Sep 23, 2024

  1. Merge branch 'master' into 618-bad-random-value

    jonbrenas authored Sep 23, 2024
    Copy the full SHA
    edebc2b View commit details
  2. Merge pull request #619 from malariagen/618-bad-random-value

    Random error when performing tests
    jonbrenas authored Sep 23, 2024
    Copy the full SHA
    55432a5 View commit details
  3. Fix center and zoom map parameters to allow int or float values (#625)

    * Upgrade center, zoom map params to floats
    
    * Explicitly allow either int, float for center, zoom map params
    leehart authored Sep 23, 2024
    Copy the full SHA
    f0b3592 View commit details
  4. Set auto_pan False for marker popup in plot_frequencies_map_markers() (

    leehart authored Sep 23, 2024
    Copy the full SHA
    6c3554b View commit details
  5. Improve performance when accessing biallelic SNP calls (#623)

    * fix bug with biallelic snp calls and variant_allele
    
    * optimisations, wip
    
    * massive thrash
    
    * comment
    
    * refactor
    
    * deal with strange performance issue in zarr and fsspec
    
    * poetry update
    
    * tweaks
    
    * revert silly mistake
    
    * avoid getattr pickle black hole of doom
    
    * fix gcs bucket
    
    * remove outputs
    
    * tweak test
    
    * tweak test
    alimanfoo authored Sep 23, 2024
    Copy the full SHA
    9f13fb0 View commit details

Commits on Sep 24, 2024

  1. Further work on scalability for large biallelic genotype data computa…

    …tions (#626)
    
    * back out usage of zarr
    
    * revert to native chunks for now
    
    * revert to native chunks for now
    alimanfoo authored Sep 24, 2024
    Copy the full SHA
    b4a3cc9 View commit details

Commits on Sep 26, 2024

  1. Copy the full SHA
    e483138 View commit details
  2. Copy the full SHA
    ee67687 View commit details
  3. Add test_snp_calls_with_sample_query_options_param

    leehart committed Sep 26, 2024
    Copy the full SHA
    0e45c8e View commit details
  4. Ensure sample_query_options is dict before unpacking

    leehart committed Sep 26, 2024
    Copy the full SHA
    ed83b3c View commit details

Commits on Sep 27, 2024

  1. Copy the full SHA
    6b25aa3 View commit details
  2. Add tests for CNV data sample_query_options

    leehart committed Sep 27, 2024
    Copy the full SHA
    64cb7d6 View commit details
Showing with 19,494 additions and 18,706 deletions.
  1. +5 −8 .github/workflows/coverage.yml
  2. +9 −12 .github/workflows/{legacy_tests.yml → integration_tests.yml}
  3. +18 −0 .github/workflows/label_issues.yml
  4. +13 −5 .github/workflows/linting.yml
  5. +6 −9 .github/workflows/notebooks.yml
  6. +2 −2 .github/workflows/tests.yml
  7. +1 −0 MANIFEST.in
  8. +11 −0 README.md
  9. +9 −1 docs/source/Af1.rst
  10. +17 −1 docs/source/Ag3.rst
  11. +1 −1 docs/source/Amin1.rst
  12. +7 −57 docs/source/_static/switcher.json
  13. +1 −1 docs/source/conf.py
  14. +1 −0 malariagen_data/__init__.py
  15. +3 −3 malariagen_data/af1.py
  16. +97 −4 malariagen_data/ag3.py
  17. +20 −3 malariagen_data/anoph/aim_data.py
  18. +42 −6 malariagen_data/anoph/base.py
  19. +32 −10 malariagen_data/anoph/base_params.py
  20. +73 −14 malariagen_data/anoph/cnv_data.py
  21. +698 −0 malariagen_data/anoph/cnv_frq.py
  22. +3 −0 malariagen_data/anoph/cnv_params.py
  23. +61 −6 malariagen_data/anoph/dipclust.py
  24. +1 −1 malariagen_data/anoph/dipclust_params.py
  25. +0 −12 malariagen_data/anoph/diplotype_distance_params.py
  26. +576 −0 malariagen_data/anoph/distance.py
  27. +66 −0 malariagen_data/anoph/distance_params.py
  28. +11 −1 malariagen_data/anoph/frq_params.py
  29. +28 −7 malariagen_data/anoph/fst.py
  30. +5 −1 malariagen_data/anoph/fst_params.py
  31. +30 −9 malariagen_data/anoph/g123.py
  32. +103 −4 malariagen_data/anoph/genome_features.py
  33. +1 −1 malariagen_data/anoph/genome_sequence.py
  34. +29 −2 malariagen_data/anoph/gplt_params.py
  35. +385 −30 malariagen_data/anoph/h12.py
  36. +11 −1 malariagen_data/anoph/h12_params.py
  37. +63 −21 malariagen_data/anoph/h1x.py
  38. +18 −4 malariagen_data/anoph/hap_data.py
  39. +277 −0 malariagen_data/anoph/hap_frq.py
  40. +22 −2 malariagen_data/anoph/hapclust.py
  41. +7 −1 malariagen_data/anoph/het_params.py
  42. +2 −2 malariagen_data/anoph/map_params.py
  43. +46 −6 malariagen_data/anoph/pca.py
  44. +60 −2 malariagen_data/anoph/pca_params.py
  45. +18 −0 malariagen_data/anoph/plink_params.py
  46. +30 −0 malariagen_data/anoph/plotly_params.py
  47. +286 −15 malariagen_data/anoph/sample_metadata.py
  48. +151 −44 malariagen_data/anoph/snp_data.py
  49. +97 −134 malariagen_data/anoph/snp_frq.py
  50. +132 −0 malariagen_data/anoph/to_plink.py
  51. +149 −1,225 malariagen_data/anopheles.py
  52. +43 −0 malariagen_data/pf8.py
  53. +118 −0 malariagen_data/pf8_config.json
  54. +1 −1 malariagen_data/plasmodium.py
  55. +1,081 −0 malariagen_data/resources/karyotype_tag_snps.csv
  56. +431 −132 malariagen_data/util.py
  57. +224 −0 notebooks/auto_chunks.ipynb
  58. +258 −0 notebooks/biallelic_snps.ipynb
  59. +8 −1 notebooks/extra_metadata.ipynb
  60. +8 −1 notebooks/igv.ipynb
  61. +399 −0 notebooks/karyotype.ipynb
  62. +121 −0 notebooks/local_cluster.ipynb
  63. +50 −0 notebooks/plink_convert.ipynb
  64. +9 −2 notebooks/plot_aims.ipynb
  65. +9 −2 notebooks/plot_cnv.ipynb
  66. +6,058 −7 notebooks/plot_diplotype_clustering.ipynb
  67. +9 −2 notebooks/plot_diversity.ipynb
  68. +48 −3 notebooks/plot_frequencies_heatmap.ipynb
  69. +309 −10,624 notebooks/plot_frequencies_space_time.ipynb
  70. +10 −3 notebooks/plot_fst_gwss.ipynb
  71. +10 −3 notebooks/plot_g123_gwss.ipynb
  72. +66 −4 notebooks/plot_genes.ipynb
  73. +205 −22 notebooks/plot_h12_h1x.ipynb
  74. +10 −3 notebooks/plot_haplotype_clustering.ipynb
  75. +10 −3 notebooks/plot_haplotype_networks.ipynb
  76. +138 −0 notebooks/plot_haplotypes_frequencies.ipynb
  77. +10 −3 notebooks/plot_heterozygosity.ipynb
  78. +10 −3 notebooks/plot_ihs_gwss.ipynb
  79. +41 −2 notebooks/plot_njt.ipynb
  80. +237 −6 notebooks/plot_pca.ipynb
  81. +86 −7 notebooks/plot_samples.ipynb
  82. +32 −4 notebooks/plot_snps.ipynb
  83. +10 −3 notebooks/plot_xpehh_gwss.ipynb
  84. +8 −1 notebooks/repr.ipynb
  85. +61 −3,413 notebooks/spike_sim_test_data.ipynb
  86. +2,559 −1,697 poetry.lock
  87. +16 −16 pyproject.toml
  88. +14 −14 tests/anoph/conftest.py
  89. 0 ...er_us_central1}/v1.0/metadata/cohorts_20221129/1229-VO-GH-DADZIE-VMF00095/samples.admin_units.csv
  90. 0 ...master_us_central1}/v1.0/metadata/cohorts_20221129/1229-VO-GH-DADZIE-VMF00095/samples.cohorts.csv
  91. 0 ...se_master_us_central1}/v1.0/metadata/cohorts_20221129/1229-VO-GH-DADZIE-VMF00095/samples.taxa.csv
  92. 0 ..._us_central1}/v1.0/metadata/cohorts_20221129/1230-VO-GA-CF-AYALA-VMF00045/samples.admin_units.csv
  93. 0 ...ster_us_central1}/v1.0/metadata/cohorts_20221129/1230-VO-GA-CF-AYALA-VMF00045/samples.cohorts.csv
  94. 0 ..._master_us_central1}/v1.0/metadata/cohorts_20221129/1230-VO-GA-CF-AYALA-VMF00045/samples.taxa.csv
  95. 0 ...us_central1}/v1.0/metadata/cohorts_20221129/1231-VO-MULTI-WONDJI-VMF00043/samples.admin_units.csv
  96. 0 ...ter_us_central1}/v1.0/metadata/cohorts_20221129/1231-VO-MULTI-WONDJI-VMF00043/samples.cohorts.csv
  97. 0 ...master_us_central1}/v1.0/metadata/cohorts_20221129/1231-VO-MULTI-WONDJI-VMF00043/samples.taxa.csv
  98. 0 ...lease_master_us_central1}/v1.0/metadata/curation/1229-VO-GH-DADZIE-VMF00095/sequence_qc_stats.csv
  99. 0 ...ase_master_us_central1}/v1.0/metadata/curation/1230-VO-GA-CF-AYALA-VMF00045/sequence_qc_stats.csv
  100. 0 ...se_master_us_central1}/v1.0/metadata/curation/1231-VO-MULTI-WONDJI-VMF00043/sequence_qc_stats.csv
  101. 0 ...fun_release_master_us_central1}/v1.0/metadata/general/1229-VO-GH-DADZIE-VMF00095/samples.meta.csv
  102. 0 ...lease_master_us_central1}/v1.0/metadata/general/1229-VO-GH-DADZIE-VMF00095/wgs_accession_data.csv
  103. 0 ...fun_release_master_us_central1}/v1.0/metadata/general/1229-VO-GH-DADZIE-VMF00095/wgs_snp_data.csv
  104. 0 ...n_release_master_us_central1}/v1.0/metadata/general/1230-VO-GA-CF-AYALA-VMF00045/samples.meta.csv
  105. 0 ...ase_master_us_central1}/v1.0/metadata/general/1230-VO-GA-CF-AYALA-VMF00045/wgs_accession_data.csv
  106. 0 ...n_release_master_us_central1}/v1.0/metadata/general/1230-VO-GA-CF-AYALA-VMF00045/wgs_snp_data.csv
  107. 0 ..._release_master_us_central1}/v1.0/metadata/general/1231-VO-MULTI-WONDJI-VMF00043/samples.meta.csv
  108. 0 ...se_master_us_central1}/v1.0/metadata/general/1231-VO-MULTI-WONDJI-VMF00043/wgs_accession_data.csv
  109. 0 ..._release_master_us_central1}/v1.0/metadata/general/1231-VO-MULTI-WONDJI-VMF00043/wgs_snp_data.csv
  110. 0 tests/anoph/fixture/{vo_agam_release → vo_agam_release_master_us_central1}/README.md
  111. 0 ...aster_us_central1}/v3.1/metadata/cohorts_20230516/1177-VO-ML-LEHMANN-VMF00004/samples.cohorts.csv
  112. 0 ...ease_master_us_central1}/v3.1/metadata/curation/1177-VO-ML-LEHMANN-VMF00004/sequence_qc_stats.csv
  113. 0 ...am_release_master_us_central1}/v3.1/metadata/general/1177-VO-ML-LEHMANN-VMF00004/samples.meta.csv
  114. 0 ...ease_master_us_central1}/v3.1/metadata/general/1177-VO-ML-LEHMANN-VMF00004/wgs_accession_data.csv
  115. 0 ...am_release_master_us_central1}/v3.1/metadata/general/1177-VO-ML-LEHMANN-VMF00004/wgs_snp_data.csv
  116. 0 ...al1}/v3.1/metadata/species_calls_aim_20220528/1177-VO-ML-LEHMANN-VMF00004/samples.species_aim.csv
  117. 0 ..._agam_release_master_us_central1}/v3/metadata/cohorts_20230516/AG1000G-AO/samples.admin_units.csv
  118. 0 ...→ vo_agam_release_master_us_central1}/v3/metadata/cohorts_20230516/AG1000G-AO/samples.cohorts.csv
  119. 0 ...se → vo_agam_release_master_us_central1}/v3/metadata/cohorts_20230516/AG1000G-AO/samples.taxa.csv
  120. 0 ...gam_release_master_us_central1}/v3/metadata/cohorts_20230516/AG1000G-BF-A/samples.admin_units.csv
  121. 0 ...vo_agam_release_master_us_central1}/v3/metadata/cohorts_20230516/AG1000G-BF-A/samples.cohorts.csv
  122. 0 ... → vo_agam_release_master_us_central1}/v3/metadata/cohorts_20230516/AG1000G-BF-A/samples.taxa.csv
  123. 0 ...lease → vo_agam_release_master_us_central1}/v3/metadata/curation/AG1000G-AO/sequence_qc_stats.csv
  124. 0 ...ase → vo_agam_release_master_us_central1}/v3/metadata/curation/AG1000G-BF-A/sequence_qc_stats.csv
  125. 0 ...gam_release → vo_agam_release_master_us_central1}/v3/metadata/general/AG1000G-AO/samples.meta.csv
  126. 0 ...lease → vo_agam_release_master_us_central1}/v3/metadata/general/AG1000G-AO/wgs_accession_data.csv
  127. 0 ...gam_release → vo_agam_release_master_us_central1}/v3/metadata/general/AG1000G-AO/wgs_snp_data.csv
  128. 0 ...m_release → vo_agam_release_master_us_central1}/v3/metadata/general/AG1000G-BF-A/samples.meta.csv
  129. 0 ...ase → vo_agam_release_master_us_central1}/v3/metadata/general/AG1000G-BF-A/wgs_accession_data.csv
  130. 0 ...m_release → vo_agam_release_master_us_central1}/v3/metadata/general/AG1000G-BF-A/wgs_snp_data.csv
  131. 0 ...ase_master_us_central1}/v3/metadata/species_calls_aim_20220528/AG1000G-AO/samples.species_aim.csv
  132. 0 ...e_master_us_central1}/v3/metadata/species_calls_aim_20220528/AG1000G-BF-A/samples.species_aim.csv
  133. 0 ...elease → vo_agam_release_master_us_central1}/v3_cohorts/cohorts_20230516/cohorts_admin1_month.csv
  134. +174 −1 tests/anoph/test_cnv_data.py
  135. +851 −0 tests/anoph/test_cnv_frq.py
  136. +246 −0 tests/anoph/test_distance.py
  137. +27 −0 tests/anoph/test_genome_features.py
  138. +119 −4 tests/anoph/test_h12.py
  139. +6 −2 tests/anoph/test_h1x.py
  140. +49 −1 tests/anoph/test_hap_data.py
  141. +193 −0 tests/anoph/test_hap_frq.py
  142. +130 −2 tests/anoph/test_pca.py
  143. +138 −0 tests/anoph/test_plink_converter.py
  144. +30 −0 tests/anoph/test_sample_metadata.py
  145. +263 −11 tests/anoph/test_snp_data.py
  146. +209 −2 tests/anoph/test_snp_frq.py
  147. +77 −0 tests/integration/test_af1.py
  148. +177 −0 tests/integration/test_ag3.py
  149. 0 tests/{ → integration}/test_amin1.py
  150. 0 tests/{test_pf7_integration.py → integration/test_pf7.py}
  151. +349 −0 tests/integration/test_pf8.py
  152. 0 tests/{test_pv4_integration.py → integration/test_pv4.py}
  153. +0 −167 tests/test_af1.py
  154. +0 −853 tests/test_ag3.py
  155. +4 −12 tests/test_plasmodium.py
  156. +1 −1 tests/test_plasmodium_config.json
13 changes: 5 additions & 8 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
@@ -15,20 +15,17 @@ jobs:
- name: Checkout source
uses: actions/checkout@v4

- name: Install poetry
run: pipx install poetry==1.8.3

- name: Setup python
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'poetry'
cache: 'pip'

- name: Install dependencies
run: poetry install
- name: Install package
run: pip install .[dev]

- name: Run tests with coverage
run: poetry run pytest --durations=20 -v --cov malariagen_data/anoph --cov-report=xml tests/anoph
- name: Run unit tests with coverage
run: pytest -v tests --ignore tests/integration --cov malariagen_data/anoph --cov-report=xml

- name: Upload coverage report
uses: codecov/codecov-action@v3
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: legacy_tests
name: integration_tests
on:
push:
branches:
@@ -7,7 +7,7 @@ on:
branches:
- master
jobs:
legacy_tests:
integration_tests:
strategy:
fail-fast: true
matrix:
@@ -23,17 +23,14 @@ jobs:
- name: Checkout source
uses: actions/checkout@v4

- name: Install poetry
run: pipx install poetry==1.8.3

- name: Setup python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "poetry"
cache: "pip"

- name: Install dependencies
run: poetry install
- name: Install package
run: pip install .[dev]

- id: 'auth'
name: 'Set up Google Cloud authentication'
@@ -53,14 +50,14 @@ jobs:
uses: actions/cache/restore@v3
with:
path: gcs_cache
key: gcs_cache_tests_20240324
key: gcs_cache_integration_tests_20240922

- name: Run full test suite
run: poetry run pytest --durations=20 --ignore=tests/anoph -v tests
- name: Run integration tests
run: pytest --durations=20 -v tests/integration

- name: Save GCS cache
uses: actions/cache/save@v3
if: always()
with:
path: gcs_cache
key: gcs_cache_tests_20240324
key: gcs_cache_integration_tests_20240922
18 changes: 18 additions & 0 deletions .github/workflows/label_issues.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Label issues
on:
issues:
types:
- reopened
- opened
jobs:
label_issues:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- run: gh issue edit "$NUMBER" --add-label "$LABELS"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
NUMBER: ${{ github.event.issue.number }}
LABELS: triage
18 changes: 13 additions & 5 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
@@ -12,13 +12,21 @@ jobs:
fail-fast: true
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- name: Checkout source
uses: actions/checkout@v4

- name: Setup python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- uses: pre-commit/action@v3.0.0
- name: Install mypy
run: pip install mypy
cache: 'pip'

- name: Run pre-commit checks
uses: pre-commit/action@v3.0.0

- name: Install package
run: pip install .[dev]

- name: Run mypy
run: |
mypy malariagen_data tests --ignore-missing-imports
15 changes: 6 additions & 9 deletions .github/workflows/notebooks.yml
Original file line number Diff line number Diff line change
@@ -19,17 +19,14 @@ jobs:
- name: Checkout source
uses: actions/checkout@v4

- name: Install poetry
run: pipx install poetry==1.8.3

- name: Setup python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "poetry"
cache: "pip"

- name: Install dependencies
run: poetry install
- name: Install package
run: pip install .[dev]

- id: 'auth'
name: 'Set up Google Cloud authentication'
@@ -49,14 +46,14 @@ jobs:
uses: actions/cache/restore@v3
with:
path: gcs_cache
key: gcs_cache_notebooks_20240324
key: gcs_cache_notebooks_20240922

- name: Run notebooks
run: poetry run jupyter nbconvert --execute notebooks/*.ipynb --inplace
run: jupyter nbconvert --execute notebooks/*.ipynb --inplace

- name: Save GCS cache
uses: actions/cache/save@v3
if: always()
with:
path: gcs_cache
key: gcs_cache_notebooks_20240324
key: gcs_cache_notebooks_20240922
4 changes: 2 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -32,5 +32,5 @@ jobs:
- name: Install package
run: pip install "${{ matrix.numpy-version }}" .[dev]

- name: Run fast unit tests
run: pytest -v tests/anoph --typeguard-packages=malariagen_data,malariagen_data.anoph
- name: Run unit tests
run: pytest -v tests --ignore tests/integration --typeguard-packages=malariagen_data,malariagen_data.anoph
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include malariagen_data/resources/*
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -127,3 +127,14 @@ the documentation via GitHub Actions.

The version switcher for the documentation can then be updated by
modifying the `docs/source/_static/switcher.json` file accordingly.

## Sponsorship

This project is currently supported by the following grants:

* [BMGF INV-068808](https://www.gatesfoundation.org/about/committed-grants/2024/04/inv-068808)
* [BMGF INV-062921](https://www.gatesfoundation.org/about/committed-grants/2024/07/inv-062921)

This project was previously supported by the following grants:

* [BMGF INV-001927](https://www.gatesfoundation.org/about/committed-grants/2019/11/inv001927)
10 changes: 9 additions & 1 deletion docs/source/Af1.rst
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ All the functions below can then be accessed as methods on the ``af1`` object. E

df_samples = af1.sample_metadata()

For more information about the data and terns of use, please see the
For more information about the data and terms of use, please see the
`MalariaGEN Anopheles funestus genomic surveillance project <https://www.malariagen.net/projects/anopheles-funestus-genomic-surveillance-project>`_
home page.

@@ -53,6 +53,8 @@ Sample metadata access
count_samples
plot_samples_bar
plot_samples_interactive_map
plot_sample_location_mapbox
plot_sample_location_geo
wgs_data_catalog
cohorts

@@ -69,6 +71,7 @@ SNP data access
is_accessible
biallelic_snp_calls
biallelic_diplotypes
biallelic_snps_to_plink

Haplotype data access
---------------------
@@ -112,6 +115,8 @@ SNP and CNV frequency analysis
aa_allele_frequencies_advanced
gene_cnv_frequencies
gene_cnv_frequencies_advanced
haplotypes_frequencies
haplotypes_frequencies_advanced
plot_frequencies_heatmap
plot_frequencies_time_series
plot_frequencies_interactive_map
@@ -132,6 +137,7 @@ Genetic distance and neighbour-joining trees (NJT)
:toctree: generated/

plot_njt
njt
biallelic_diplotype_pairwise_distances

Heterozygosity analysis
@@ -161,6 +167,8 @@ Genome-wide selection scans
plot_h12_calibration
h12_gwss
plot_h12_gwss
plot_h12_gwss_multi_panel
plot_h12_gwss_multi_overlay
h1x_gwss
plot_h1x_gwss
g123_calibration
18 changes: 17 additions & 1 deletion docs/source/Ag3.rst
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ All the functions below can then be accessed as methods on the ``ag3`` object. E

df_samples = ag3.sample_metadata()

For more information about the data and terns of use, please see the
For more information about the data and terms of use, please see the
`MalariaGEN Anopheles gambiae genomic surveillance project <https://www.malariagen.net/anopheles-gambiae-genomic-surveillance-project>`_
home page.

@@ -49,10 +49,13 @@ Sample metadata access
sample_metadata
add_extra_metadata
clear_extra_metadata
cross_metadata
count_samples
lookup_sample
plot_samples_bar
plot_samples_interactive_map
plot_sample_location_mapbox
plot_sample_location_geo
wgs_data_catalog
cohorts

@@ -69,6 +72,7 @@ SNP data access
is_accessible
biallelic_snp_calls
biallelic_diplotypes
biallelic_snps_to_plink

Haplotype data access
---------------------
@@ -121,6 +125,8 @@ SNP and CNV frequency analysis
aa_allele_frequencies_advanced
gene_cnv_frequencies
gene_cnv_frequencies_advanced
haplotypes_frequencies
haplotypes_frequencies_advanced
plot_frequencies_heatmap
plot_frequencies_time_series
plot_frequencies_interactive_map
@@ -141,6 +147,7 @@ Genetic distance and neighbour-joining trees (NJT)
:toctree: generated/

plot_njt
njt
biallelic_diplotype_pairwise_distances

Heterozygosity analysis
@@ -170,6 +177,8 @@ Genome-wide selection scans
plot_h12_calibration
h12_gwss
plot_h12_gwss
plot_h12_gwss_multi_panel
plot_h12_gwss_multi_overlay
h1x_gwss
plot_h1x_gwss
g123_calibration
@@ -208,3 +217,10 @@ Fst analysis
plot_pairwise_average_fst
fst_gwss
plot_fst_gwss

Inversion karyotypes
--------------------
.. autosummary::
:toctree: generated/

karyotype
2 changes: 1 addition & 1 deletion docs/source/Amin1.rst
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ All the functions below can then be accessed as methods on the ``amin1`` object.

df_samples = amin1.sample_metadata()

For more information about the data and terns of use, please see the
For more information about the data and terms of use, please see the
`MalariaGEN Vector Observatory Asia <https://www.malariagen.net/mosquito/vector-observatory-asia>`_
home page.

Loading