Merge pull request #23 from BU-ISCIII/develop

Release 3.0.0

.github/CONTRIBUTING.md

@@ -0,0 +1,72 @@
+# taranys: Contributing Guidelines
+
+## Contribution workflow
+
+If you'd like to write or modify some code for taranys, the standard workflow is as follows:
+
+1. Check that there isn't already an issue about your idea in the [taranys issues](https://github.com/BU-ISCIII/taranys/issues) to avoid duplicating work. **If there isn't one already, please create one so that others know you're working on this**.
+2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [taranys repository](https://github.com/BU-ISCIII/taranys/) to your GitHub account.
+3. Make the necessary changes / additions within your forked repository following the [code style guidelines](#code-style-guidelines).
+4. Modify the [`CHANGELOG`](../CHANGELOG.md) file according to your changes in the appropiate section ([X.X.Xdev]), you should register your changes regarding:
+   1. Added enhancements
+   2. New modules
+   3. Fixes
+   4. Removed stuff
+   5. Requirements added or version update
+5. Update any documentation as needed.
+6. [Submit a Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) against the `develop` branch and send the url to the #pipelines-dev channel in slack (if you are not in the slack channel just wait fot the PR to be reviewed and rebased).
+
+If you're not used to this workflow with git, you can start with:
+
+- Some [docs in the bu-isciii wiki](https://github.com/BU-ISCIII/BU-ISCIII/wiki/Github--gitflow).
+- [some slides](https://docs.google.com/presentation/d/1PruqGxPQVxtNcuEbOd86mylXorgYIU5a/edit?pli=1#slide=id.p1) (in spanish).
+- some github generic docs [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests).
+- even their [excellent `git` resources](https://try.github.io/).
+
+### taranys repo branches
+
+taranys repo works with a two branching scheme: `main` and `develop`.
+
+- `main`: stable code only for releases.
+- `develop`: new code development for the different modules.
+
+You need to submit your PR always against `develop`. Once approbed, this changes must be **`rebased`** so we do not create empty unwanted merges.
+
+## Tests
+
+When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
+Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
+
+There are typically two types of tests that run:
+
+### Lint tests
+
+We use black and flake8 linting based on PEP8 guidelines for python coding. You can check more information [here](https://github.com/BU-ISCIII/BU-ISCIII/wiki/Python#linting).
+
+### Code tests
+
+Taranys modules are executed using a test dataset.
+
+Anyhow you should always submit locally tested code!!
+
+### New version bumping and release
+
+In order to create a new release you need to follow the next steps:
+
+1. Set the new version according to [semantic versioning](https://semver.org/), in our particular case, changes in the `hotfix` branch will change the PATCH version (third one), and changes in develop will typicaly change the MINOR version, unless the developing team decides otherwise.
+2. Create a PR bumping the new version against `develop`. For bumping a new version just change [this line](https://github.com/BU-ISCIII/taranys/blob/09c00c1ddd11f7489de7757841aff506ef4b7e1d/setup.py#L5) with the new version.
+3. Once that PR is merged, create via web another PR against `main` (origin `develop`). This PR would need 2 approvals.
+4. [Create a new release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository) copying the appropiate notes from the `CHANGELOG`.
+5. Once the release is approved and merged, you're all set!
+
+PRs from one branch to another, like in a release should be **`merged`** not rebased, so we avoid conflicts and the branch merge is correctly visualize in the commits history.
+
+### Code style guidelines
+
+We follow PEP8 conventions as code style guidelines, please check [here](https://github.com/BU-ISCIII/BU-ISCIII/wiki/Python#pep-8-guidelines-read-the-full-pep-8-documentation) for more detail.
+
+When developing new code, we strongly recommend to implement LogSum() functions from log_summary.py instead of the classic python logging in order to keep track of all the warnings and errors that may appear during any of the processes.
+
+## Getting help
+
+For further information/help, please ask on the  `#pipelines-dev` slack channel or write us an email! ([[email protected]](emailto:[email protected])).

.github/workflows/pypi_publish.yml

@@ -0,0 +1,49 @@
+name: Publish package python distribution to Pypi
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+
+jobs:
+    build:
+      name: Build distribution
+      runs-on: ubuntu-latest
+      steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.12.7
+      - name: Install pypi/build
+        run: >-
+          python3 -m
+          pip install
+          build
+          --user
+      - name: Build a binary wheel and a source tarball
+        run: python3 -m build
+      - name: Store the distribution packages
+        uses: actions/upload-artifact@v4
+        with:
+          name: python-package-distributions
+          path: dist/
+
+    publish-to-pypi:
+      name: Publish dist to PyPI
+      needs:
+      - build
+      runs-on: ubuntu-latest
+      environment:
+        name: pypi
+        url: https://pypi.org/p/taranys
+      permissions:
+        id-token: write
+      steps:
+      - name: Download all the dists
+        uses: actions/download-artifact@v4
+        with:
+          name: python-package-distributions
+          path: dist/
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

.github/workflows/python_lint.yml

@@ -0,0 +1,35 @@
+name: python_lint
+
+on:
+  push:
+    paths:
+      - '**.py'
+  pull_request:
+    paths:
+      - '**.py'
+
+jobs:
+  flake8_py3:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Setup Python
+        uses: actions/setup-python@v1
+        with:
+          python-version: 3.9.x
+          architecture: x64
+      - name: Checkout PyTorch
+        uses: actions/checkout@master
+      - name: Install flake8
+        run: pip install flake8
+      - name: Run flake8
+        run: flake8 --ignore E501,W503,E203,W605
+
+  black_lint:
+    runs-on: ubuntu-latest
+    steps:
+        - name: Setup
+          uses: actions/checkout@v2
+        - name: Install black in jupyter
+          run: pip install black[jupyter]
+        - name: Check code lints with Black
+          uses: psf/black@stable

.github/workflows/tests.yml

@@ -1,38 +1,55 @@
 name: tests ci
-# This workflow runs the pipeline with the minimal test dataset to check that it completes any errors
+# This workflow runs the pipeline with the minimal test dataset to check 
+# is completed without any errors
 on:
-  push:
-    branches: [develop]
-  pull_request_target:
-    branches: [develop]
-  release:
-    types: [published]
+    pull_request:
+    push:
 
 jobs:
-  push_dockerhub:
-    name: Push new Docker image to Docker Hub (dev)
+  test_taranys:
     runs-on: ubuntu-latest
-    # Only run for the official repo, for releases and merged PRs
-    if: ${{ github.repository == 'BU-ISCIII/taranis' }}
-    env:
-      DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
-      DOCKERHUB_PASS: ${{ secrets.DOCKERHUB_PASSWORD }}
+
     steps:
-      - name: Check out pipeline code
-        uses: actions/checkout@v2
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Miniconda
+        uses: conda-incubator/setup-miniconda@v3
+        with:
+          activate-environment: taranys_env
+          environment-file: environment.yml
 
-      - name: Build new docker image
-        run: docker build --no-cache . -t buisciii/taranis:dev
+      - name: Verify conda environment
+        run: conda env list
 
-      - name: Push Docker image to DockerHub (develop)
+      - name: Activate env and install taranys
         run: |
-          echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
-          docker push buisciii/taranis:dev
-  run-tests:
-    name: Run tests
-    needs: push_dockerhub
-    runs-on: ubuntu-latest
-    steps:
-      - name: Run pipeline with test data
+          source $CONDA/etc/profile.d/conda.sh
+          conda activate taranys_env
+          python -m pip install .
+          
+      - name: Testing analyze schema allele
+        run: |
+          source $CONDA/etc/profile.d/conda.sh
+          conda activate taranys_env
+          taranys analyze-schema -i  test/MLST_listeria/analyze_schema -o analyze_schema_test  --cpus 1 --output-allele-annot --remove-no-cds --remove-duplicated --remove-subset
+
+      - name: Testing reference allele
+        run: |
+          source $CONDA/etc/profile.d/conda.sh
+          conda activate taranys_env
+          taranys reference-alleles -s test/MLST_listeria/reference_allele -o reference_allele_test --cpus 1 
+          
+      - name: Testing allele calling
+        run: |
+          source $CONDA/etc/profile.d/conda.sh
+          conda activate taranys_env
+          taranys reference-alleles -s analyze_schema_test/new_schema -o reference_allele --cpus 1 
+          taranys allele-calling --force --schema analyze_schema_test/new_schema --reference reference_allele --annotation analyze_schema_test/allele_annotation.tar.gz --output allele_calling_test --cpus 1 --snp --alignment test/samples_listeria/*.fasta
+
+      - name: Testing distance matrix
         run: |
-            docker run buisciii/taranis:dev bash -c /opt/taranis/test/test.sh
+            source $CONDA/etc/profile.d/conda.sh
+            conda activate taranys_env
+            mkdir distance_matrix_result
+            taranys distance-matrix --alleles allele_calling_test/allele_calling_match.csv --force --output distance_matrix_result

.gitignore

@@ -9,6 +9,7 @@ __pycache__/
 *.log.*
 # Projects files
 *.komodoproject
+*.vscode/
 
 # Distribution / packaging
 .Python
@@ -29,6 +30,7 @@ wheels/
 .installed.cfg
 *.egg
 MANIFEST
+poetry.lock
 
 # PyInstaller
 #  Usually these files are written by a python script from a template
@@ -90,6 +92,7 @@ venv/
 ENV/
 env.bak/
 venv.bak/
+virtualenv/
 
 # Spyder project settings
 .spyderproject

.readthedocs.yaml

@@ -0,0 +1,19 @@
+# Read the Docs configuration file for MkDocs projects
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
+mkdocs:
+  configuration: mkdocs.yml
+
+# Optionally declare the Python requirements required to build your docs
+python:
+  install:
+  - requirements: docs/requirements.txt