Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/refactor kinship print to write #43

Open
wants to merge 47 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
73e7f37
add pandas to dockerfile
ellendejong Aug 20, 2024
4d22c20
refactor kinship: use pandas and add user friendly message if kinship…
ellendejong Aug 20, 2024
9059f93
Install CustomModules as package to enable import of utils
ellendejong Aug 27, 2024
658a832
add non_empty_existing_path to utils
ellendejong Aug 27, 2024
0e9d1f7
Move parse_ped to utils and add unit tests
ellendejong Aug 27, 2024
c56a676
add utils and remove unused imports
ellendejong Aug 27, 2024
2857d93
use non_empty_existing_path in argparser
ellendejong Aug 27, 2024
3b7909f
fix quote issues
ellendejong Aug 27, 2024
16abdac
add pytest
ellendejong Aug 27, 2024
677032f
add docstrings to check_kinship
ellendejong Aug 27, 2024
0d7c4d1
add docstring to non_empty_existing_path
ellendejong Aug 27, 2024
c5c9d13
add docstring parse_ped
ellendejong Aug 27, 2024
0d10f6e
resolve flake8 warnings and errors.
ellendejong Aug 27, 2024
0e6a209
add pyproject custommodules
ellendejong Sep 18, 2024
96f0bba
Instead of using util functions, duplicate function in tool.
ellendejong Sep 20, 2024
f4b5d49
add poetry project
ellendejong Sep 20, 2024
2a7d6ac
Add github workflows kinship lint and test
ellendejong Sep 20, 2024
bcffc89
Change python version
ellendejong Sep 20, 2024
407e569
Add missing dependencies using poetry
ellendejong Sep 20, 2024
81ea36e
add dependency pytest-datadir
ellendejong Sep 20, 2024
1d7f5dc
remove custommodules install as package in readme
ellendejong Sep 20, 2024
e22c2d6
remove init in custommodules
ellendejong Sep 20, 2024
79cd433
revert parse_ped pytest in utils, will be added to dev via another br…
ellendejong Sep 20, 2024
4fc6e20
typo - vs _
ellendejong Sep 16, 2024
1e05063
open file in parse_ped
ellendejong Sep 17, 2024
b185b49
add on push
ellendejong Sep 27, 2024
f5965ca
add working dir to ruff
ellendejong Sep 27, 2024
7e47f74
Filter paths as list
ellendejong Oct 4, 2024
b8d54cd
add poetry group dev dependencies
ellendejong Oct 4, 2024
dcf41b4
Add author
ellendejong Oct 4, 2024
986175e
add Authors
ellendejong Oct 4, 2024
2066991
poetry group dev dependencies
ellendejong Oct 4, 2024
2c309fe
revert changes
ellendejong Oct 7, 2024
2bd6ec0
Use poetry to install dependencies in docker file.
ellendejong Oct 7, 2024
11c12bf
Increase poetry tool version.
ellendejong Oct 7, 2024
35f45ed
Change function name read_kinship to read_and_modify_kinship
ellendejong Oct 7, 2024
c7f01aa
Delete poetry.lock and pyproject.toml from CustomModules dir
ellendejong Oct 7, 2024
fd7492a
Add final else.
ellendejong Oct 7, 2024
b67849b
Add 'validate' to function name.
ellendejong Oct 7, 2024
479d897
Add try-except when reading ped file.
ellendejong Oct 7, 2024
eb4ab75
Use 'is' in type comparisons
ellendejong Oct 7, 2024
18b7a19
Run pytest with poetry.
ellendejong Oct 9, 2024
3d80afa
Specify github action poetry version.
ellendejong Oct 9, 2024
d6c3c02
Dockerfile; poetry installation in its own isolated venv
ellendejong Oct 9, 2024
0d3af94
Add VIRTUAL_ENV.
ellendejong Oct 9, 2024
da15079
use pip3
ellendejong Oct 11, 2024
3f4ef68
Remove setuptools.
ellendejong Oct 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/kinship_lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Kinship Lint
on:
pull_request:
paths: [Kinship/**]
push:
branches: [master, develop]
ellendejong marked this conversation as resolved.
Show resolved Hide resolved
paths: [Kinship/**]

jobs:
ruff:
runs-on: ubuntu-latest
defaults:
run:
working-directory: Kinship/
steps:
- uses: actions/checkout@v4
- uses: chartboost/ruff-action@v1
63 changes: 63 additions & 0 deletions .github/workflows/kinship_test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Source: https://github.com/marketplace/actions/install-poetry-action
name: Kinship Test
on:
pull_request:
paths: [Kinship/**]
push:
branches: [master, develop]
ellendejong marked this conversation as resolved.
Show resolved Hide resolved
paths: [Kinship/**]

jobs:
pytest:
runs-on: ubuntu-latest
defaults:
run:
working-directory: Kinship/
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v4
- name: Set up python
id: setup-python
uses: actions/setup-python@v5
with:
python-version: '3.11.5'
#----------------------------------------------
# install & configure poetry
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
ellendejong marked this conversation as resolved.
Show resolved Hide resolved
with:
version: 1.8.3
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv_kinship-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
#----------------------------------------------
# install root project
#----------------------------------------------
- name: Install project
run: poetry install --no-interaction
#----------------------------------------------
# run pytest
#----------------------------------------------
- name: Run tests
run: |
poetry run pytest .
42 changes: 38 additions & 4 deletions Kinship/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,44 @@
FROM --platform=linux/amd64 continuumio/miniconda3
################## BASE IMAGEs ######################
FROM continuumio/miniconda3:24.5.0-0

LABEL base_image="continuumio/miniconda3"
LABEL version="1.0.0"
LABEL extra.binaries="vcftools, plink, king"
################## METADATA ######################
LABEL base_image="continuumio/miniconda3:24.5.0-0"
LABEL version="1.1.1"
LABEL extra.binaries="vcftools, plink, king, pandas"

################## INSTALLATIONS ######################
# Use conda to install other dependencies.
RUN conda config --add channels bioconda && \
conda config --add channels conda-forge && \
conda install -y vcftools=0.1.14 plink=1.90b4 king=2.2.7

# Use poetry to install virtualenv.
ENV POETRY_VERSION=1.8.3
ENV POETRY_HOME=/opt/poetry
ENV POETRY_VENV=/opt/poetry-venv

# Tell Poetry where to place its cache and virtual environment
ENV POETRY_CACHE_DIR=/tmp/poetry_cache

# Do not ask any interactive question
ENV POETRY_NO_INTERACTION=1

# Make poetry create the virtual environment in the project's root
# it gets named `.venv`
ENV POETRY_VIRTUALENVS_IN_PROJECT=1
ENV POETRY_VIRTUALENVS_CREATE=1

# Set virtual_env variable
ENV VIRTUAL_ENV=/.venv
# Prepend virtual environments path
ENV PATH="${VIRTUAL_ENV}/bin:${POETRY_VENV}/bin:${PATH}"

# Creating a virtual environment just for poetry and install it with pip
RUN python3 -m venv $POETRY_VENV \
&& $POETRY_VENV/bin/pip3 install poetry==${POETRY_VERSION}

# Copy project requirement files here to ensure they will be cached.
COPY pyproject.toml poetry.lock ./

# Install dependencies.
RUN --mount=type=cache,target=$POETRY_CACHE_DIR poetry install --without dev --no-interaction --no-cache
7 changes: 4 additions & 3 deletions Kinship/Kinship.nf
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
process Kinship {
tag {"Kinship ${analysis_id}"}
label 'Kinship'
container = 'docker.io/umcugenbioinf/kinship:1.0.0'
container = 'ghcr.io/umcugenetics/kinship:1.1.1'

shell = ['/bin/bash', '-euo', 'pipefail']

input:
Expand All @@ -17,6 +18,6 @@ process Kinship {
plink --file out --make-bed --noweb
king -b plink.bed --kinship
cp king.kin0 ${analysis_id}.kinship
python ${projectDir}/CustomModules/Kinship/check_kinship.py ${analysis_id}.kinship ${ped_file} > ${analysis_id}.kinship_check.out
python ${projectDir}/CustomModules/Kinship/check_kinship.py ${analysis_id}.kinship ${ped_file} --output_prefix ${analysis_id} --output_path .
"""
}
}
Loading