Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic Rework #47

Merged
merged 162 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
162 commits
Select commit Hold shift + click to select a range
43cb4a2
Rename underscore modules
multimeric Jul 19, 2023
7f35ac4
Merge branch 'optional-napari' into thin-wrapper
multimeric Jul 21, 2023
943fb69
Clean up global variables and types in dock_widget
multimeric Jul 21, 2023
7e3a78f
Clean up dock_widget
multimeric Jul 26, 2023
e3742c7
Fix missing imports in deconvolution
multimeric Jul 27, 2023
da2a363
Merge branch 'optional-napari' of github.com:multimeric/napari_lattic…
multimeric Jul 27, 2023
3a9f357
Merge branch 'master' of github.com:BioimageAnalysisCoreWEHI/napari_l…
multimeric Jul 28, 2023
248a645
More dock_widget cleanup
multimeric Jul 28, 2023
695b0ea
Type signature tidying, and remove an unused function
multimeric Aug 2, 2023
9752c68
Refactor workflow augmentation
multimeric Aug 4, 2023
5db9120
Object oriented system
multimeric Aug 4, 2023
e9f3866
Move to Pydantic, and add field validations for LatticeData and similar
multimeric Aug 10, 2023
ba63f39
Refactor process() to return a ProcessedSlices object
multimeric Aug 10, 2023
c02ea40
Fix breaking plugin errors
multimeric Aug 11, 2023
eee538f
Initial Restructure of GUI
multimeric Aug 16, 2023
48b971a
Finish draft of one-page GUI
multimeric Aug 17, 2023
2370be8
First pass of preview working in the GUI
multimeric Aug 21, 2023
1e41627
Implemented traffic light system for first two field sets
multimeric Sep 1, 2023
d43bce7
Refactor field validation into mixin
multimeric Sep 4, 2023
73f248e
Metaclass attempt
multimeric Sep 5, 2023
af02a25
Code cleanup, dimension stacking, use mixin for validation logic
multimeric Sep 5, 2023
28e62b9
Implement "disabled", greyed-out steps, fix some bugs in the Lattice
multimeric Sep 6, 2023
39f798b
Add new tab icons, filter down to only images in the layer select, make
multimeric Sep 7, 2023
23224b1
Rework the dimension merging
multimeric Sep 11, 2023
17e10b6
Cleanup old code
multimeric Sep 11, 2023
df6513a
Use pydantic-cli for CLI
multimeric Sep 12, 2023
652a6de
Refactor submodels into different files
multimeric Sep 14, 2023
a0e4bb1
Rewrite to use typer, all in on pydantic validators
multimeric Sep 14, 2023
4a91550
Clean up commented/unused code
multimeric Sep 14, 2023
23f2b77
Merge branch 'master' of github.com:BioimageAnalysisCoreWEHI/napari_l…
multimeric Sep 15, 2023
2d20bdf
Fix missing imports
multimeric Sep 15, 2023
5395f5b
pydantic-numpy
multimeric Sep 15, 2023
3f844c9
Use older pydantic-numpy
multimeric Sep 15, 2023
a243417
Remove pydantic-numpy
multimeric Sep 15, 2023
d9695f0
Pyright settings in the pyproject.toml
multimeric Sep 15, 2023
1b0b0f3
Rework workflow handling, move sample data into package
multimeric Sep 15, 2023
c48b074
Add pkg_resources dependency
multimeric Sep 15, 2023
61901ef
GUI fixes
multimeric Sep 15, 2023
2512401
Fix type errors
multimeric Sep 15, 2023
57f9a2c
Fix dependency issues
multimeric Sep 15, 2023
04ba1d2
Cartesian product tests
multimeric Sep 15, 2023
094651a
Fix some tests, add comprehensive LatticeData tests
multimeric Sep 15, 2023
bc006e3
Skip validations when image is missing
multimeric Sep 16, 2023
867ca38
invoke() test utility, forbid extra fields, ignore_keyerror() utility
multimeric Sep 16, 2023
0d20f0d
Add formal click depenedency
multimeric Sep 17, 2023
bb5d321
Refactor CLI to use None default values, add JSON config help, throw
multimeric Sep 20, 2023
8a4836e
Fix GUI bugs, fix some CLI tests to use subcommands
multimeric Sep 20, 2023
af36f86
Fix cli tests, use click argument panels, clean up CLI code
multimeric Sep 20, 2023
fe0d7f6
Fix CLI tests
multimeric Sep 20, 2023
8b248c8
Clean up LatticeData tests
multimeric Sep 20, 2023
b93be4f
Implement pixel data source selection
multimeric Sep 21, 2023
6113c37
Copy the args before modifying them
multimeric Sep 21, 2023
ba2f5cc
Improvements to workflow handling
multimeric Sep 22, 2023
a6a2f01
Missing test files
multimeric Sep 22, 2023
e27b19e
More missing files, disable cropping test until we have ROIs
multimeric Sep 22, 2023
02baf7b
Add missing params.py
multimeric Sep 22, 2023
b593ecd
Fix test_process_workflow test suite
multimeric Sep 22, 2023
6b856c4
Removed plugin preview
multimeric Sep 25, 2023
4202b58
Clean up GUI code, and fix tests
multimeric Sep 25, 2023
2e750d1
Fix ROI handling, add ROI class, fix plugin linting
multimeric Sep 25, 2023
cba0eb3
Fix for when crop is None
multimeric Sep 25, 2023
f61f887
Add missing ROI test data
multimeric Sep 25, 2023
ac06150
Stop using static methods
multimeric Sep 25, 2023
fa25105
Remove commented crop test
multimeric Oct 2, 2023
872da0e
Fix enable_if documentation
multimeric Oct 2, 2023
2d31b9b
Fix tab naming
multimeric Oct 2, 2023
98c65d1
Pretty print LatticeData in GUI
multimeric Oct 2, 2023
7048b6e
Stop re-running validations when napari layers change
multimeric Oct 2, 2023
49fb8b7
Fix for crop enabling, ADD_RECTANGLE for cropping, fix for ROI selection
multimeric Oct 2, 2023
b8da344
Add more checks before we load the workflow modules
multimeric Oct 2, 2023
f459cc9
Add save completed notification
multimeric Oct 2, 2023
4dc8bc1
Add rich as formal plugin dependency
multimeric Oct 2, 2023
baf0c24
Show CLI help by default
multimeric Oct 3, 2023
cef8dce
Pretty print table of validation errors in CLI
multimeric Oct 3, 2023
cdf1254
Add rich as formal CLI dependency
multimeric Oct 3, 2023
3eed0f3
Always print help when no argumeunts provided
multimeric Oct 4, 2023
cb65609
Fix tests after subcommands were removed
multimeric Oct 4, 2023
7909106
Rename image to input_image, and fix the CLI validation error display
multimeric Oct 4, 2023
98c435d
Fix for the confusing "time_range │ 'NoneType' object is not itera…
multimeric Oct 4, 2023
16b28fe
Add roi_subset parameter
multimeric Oct 4, 2023
17900d8
Fix workflow test
multimeric Oct 4, 2023
79f1365
Infer the output dir correctly, separate validation and process tests
multimeric Oct 10, 2023
4483693
Workflow loads from YAML in CLI
multimeric Oct 10, 2023
82a73b2
Fix broken CLI tests
multimeric Oct 10, 2023
404f80b
Create test_cli test suite
multimeric Oct 13, 2023
561f125
Fix save directory selection
multimeric Oct 13, 2023
8e5508e
Workflow processing into data frames
multimeric Oct 27, 2023
72be81e
Document the new IO modules, remove the old io.py
multimeric Oct 30, 2023
d037d9d
Apply feedback to plugin, change save suffix behaviour
multimeric Oct 30, 2023
e757779
Add test suite for GUI saving
multimeric Oct 30, 2023
8da6632
Fix type import errors
multimeric Oct 30, 2023
66462b3
Fix CLI tests that now need an output directory
multimeric Oct 31, 2023
8b8c4e5
Only run all the tests on one version, to avoid using excessive CPU h…
multimeric Oct 31, 2023
eba792a
${{ }} notation
multimeric Oct 31, 2023
891549c
Single quotes?
multimeric Oct 31, 2023
db52dda
Use public API for finding the tab widget
multimeric Oct 31, 2023
6cf492f
Magicgui dependency bound
multimeric Oct 31, 2023
69500a7
Fix for missing output directory for previewing; add make() method for
multimeric Nov 3, 2023
c09983f
Add validation for z range, and loosen the GUI restrictions
multimeric Nov 3, 2023
f757e53
Dynamically update the maximum range values
multimeric Nov 10, 2023
ae721cc
Clean up code
multimeric Nov 10, 2023
2000786
Remove the z cropping fields from the GUI
multimeric Nov 15, 2023
8251aca
Fix for cropping by treating roi as floats
multimeric Nov 17, 2023
60a1c15
Revert "Remove the z cropping fields from the GUI"
multimeric Nov 17, 2023
5addbcc
Tests for float ROIs
multimeric Nov 17, 2023
e36bcfd
Rework Z-range handling to use deskewed max, and to update automatically
multimeric Nov 20, 2023
9990f78
Fix broken GUI tests
multimeric Nov 20, 2023
b770b32
Fix deskew vol usage
multimeric Nov 23, 2023
ad575f5
Minimal test images
multimeric Dec 6, 2023
b2842f5
Add progress bars more consistently
multimeric Dec 6, 2023
eaa6a7b
Add tool tip for PSFs
multimeric Dec 7, 2023
42a5537
Add working parent_connect implementation
multimeric Dec 8, 2023
9d4debb
Use TupleEdit for PSFs
multimeric Dec 11, 2023
ecf62d5
Fix negative crop excess
multimeric Dec 13, 2023
611949e
GUI cropping improvements; remove PLACEHOLDER
multimeric Dec 13, 2023
979b92b
Fix for array parameters such as time_range in the CLI
multimeric Dec 14, 2023
6b0e477
Don't compute the dask array when calculating deskewed shape
multimeric Dec 18, 2023
b645597
fix voxel_size_z for cropping
pr4deepr Dec 18, 2023
0d7c7d3
Add the missing suffix parameter into the GUI save function
multimeric Dec 19, 2023
884b6b7
Merge branch 'thin-wrapper' of github.com:multimeric/napari_lattice i…
multimeric Dec 19, 2023
ef448fc
Dynamically rescale raw input image according to pixel sizes
multimeric Dec 19, 2023
b33772d
Reset view to centre the image
pr4deepr Dec 19, 2023
9adf364
Fix for crop + deconvolution, with a test
multimeric Dec 19, 2023
298c52b
Merge branch 'thin-wrapper' of github.com:multimeric/napari_lattice i…
multimeric Dec 19, 2023
1868666
Fix "test_process_deconvolution"
multimeric Dec 19, 2023
6a988a6
Reject CropParams with an empty ROI list
multimeric Dec 20, 2023
15e5d8e
Add --show-schema CLI flag, and rework the way we document CLI config
multimeric Dec 20, 2023
2a2024a
Re-enable testing on all python versions; copy a pairwise implementation
multimeric Dec 20, 2023
f26266f
Revert workflow argument handling to the old method: only passing the
multimeric Dec 21, 2023
3023154
Fix tests broken by previous commit
multimeric Dec 21, 2023
90f3523
Undo changes to workflow example
multimeric Dec 21, 2023
9e19163
Remove some dead workflow code
multimeric Jan 11, 2024
c6fb782
added function to return csv filepath
pr4deepr Jan 15, 2024
6f15bb4
avoid overwriting of output file from workflow; fix how dataframes cr…
pr4deepr Jan 16, 2024
ba51e68
revert change, and use pandas series explode instead
pr4deepr Jan 17, 2024
b294172
Update core/lls_core/models/output.py
pr4deepr Jan 18, 2024
1242927
take only dataframe for now
pr4deepr Jan 18, 2024
2139687
A pixel tuple should be in the order of Z, Y, then X
multimeric Aug 1, 2024
420898b
Use AICS metadata even when GUI isn't involved
multimeric Aug 1, 2024
b28e5ef
Override read_image validator in LatticeData to get the filename
multimeric Aug 2, 2024
f064ea0
Only use AICS pixels if they are all defined
multimeric Aug 2, 2024
3731a21
Remove the standard default for pixel sizes
multimeric Aug 2, 2024
2fb523a
Get default pixels from another model
multimeric Aug 2, 2024
d7c497e
Always run pixel validator
multimeric Aug 2, 2024
0958c8a
Separate input and output dirs for CLI tests
multimeric Aug 6, 2024
2f6ddf9
User feedback: progress total and success message
multimeric Aug 6, 2024
371287e
Get output dir from lattice
multimeric Aug 6, 2024
5d1118a
Improve message spacing, fix incorrect channel and time in preview
multimeric Aug 8, 2024
ca6e1d7
Rework progress bars to be in one place, and have separate ones for
multimeric Aug 8, 2024
f04f30e
Clean up channel progress bar
multimeric Aug 8, 2024
4dfeb7c
Fix bug when saving non-contiguous ranges
multimeric Aug 9, 2024
14627c5
Fix for crop + workflow
multimeric Aug 14, 2024
4a3791c
Add channel and time to workflow data frames
multimeric Aug 14, 2024
91c7ea9
Rework workflow handling: validation, fixed input name etc
multimeric Aug 15, 2024
2512d3c
Old python type issues
multimeric Aug 15, 2024
5f29bc7
Use "deskewed_image" in all workflow tests
multimeric Aug 16, 2024
785010e
Convert psf to numpy in crop workflow
multimeric Aug 16, 2024
3ddc78c
Workflow previews
multimeric Aug 22, 2024
35baf42
pycudadecon dependencies
multimeric Aug 22, 2024
938d2ed
Limit max napari version
multimeric Aug 22, 2024
399fd74
Add fix for CLE array output by workflow
multimeric Aug 22, 2024
b7dad44
Add binarisation workflow for testing
multimeric Aug 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions .github/workflows/plugin_preview.yml

This file was deleted.

210 changes: 105 additions & 105 deletions .github/workflows/test_and_deploy.yml
Original file line number Diff line number Diff line change
@@ -1,105 +1,105 @@
# This workflows will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
# For pytest-qt related fixes: https://pytest-qt.readthedocs.io/en/latest/troubleshooting.html#github-actions

name: tests

on:
pull_request:
paths-ignore:
- "**/README.md"
push:
paths-ignore:
- "**/README.md"

jobs:
test:
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
env:
DISPLAY: ":99.0"
steps:
- uses: actions/checkout@v3

- uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
activate-environment: test
python-version: ${{ matrix.python-version }}
channels: conda-forge

- name: Conda info
run: conda info

- name: Save conda location
run: echo "python=$(which python)" >> "$GITHUB_ENV"

- name: Install core
timeout-minutes: 10
run: |
conda install -y pyopencl pocl
python --version
pip install --upgrade pip setuptools wheel
pip install --use-pep517 -e './core[testing]'

- name: Lint core
uses: ./.github/lint
with:
directory: core
python: ${{ env.python }}

- name: Test core
run: pytest -v --cov=core --cov-report=xml core/

- uses: tlambert03/setup-qt-libs@v1

- name: Install plugin
timeout-minutes: 10
run: |
/sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1920x1200x24 -ac +extension GLX
pip install pytest-qt PyQt5
pip install -e './plugin[testing]'

- name: Lint plugin
uses: ./.github/lint
with:
directory: plugin
python: ${{ env.python }}

- name: Test plugin
run: pytest -v --cov=plugin --cov-report=xml plugin

- name: Coverage
uses: codecov/codecov-action@v3

deploy:
# this will run when you have tagged a commit, starting with "v*"
# and requires that you have put your twine API key in your
# github secrets (see readme for details)
needs: [test]
runs-on: ubuntu-latest
if: contains(github.ref, 'tags')
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.x"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -U setuptools setuptools_scm wheel twine
- name: Build and publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.TWINE_TOKEN }}
run: |
git tag
python setup.py sdist bdist_wheel
twine upload dist/*
# This workflows will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
# For pytest-qt related fixes: https://pytest-qt.readthedocs.io/en/latest/troubleshooting.html#github-actions
name: tests
on:
pull_request:
paths-ignore:
- "**/README.md"
push:
paths-ignore:
- "**/README.md"
jobs:
test:
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
env:
DISPLAY: ":99.0"
steps:
- uses: actions/checkout@v3
- uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
activate-environment: test
python-version: ${{ matrix.python-version }}
channels: conda-forge
- name: Conda info
run: conda info
- name: Save conda location
run: echo "python=$(which python)" >> "$GITHUB_ENV"
- name: Install core
timeout-minutes: 10
run: |
conda install -y pyopencl pocl
python --version
pip install --upgrade pip setuptools wheel
pip install --use-pep517 -e './core[testing]'
- name: Lint core
uses: ./.github/lint
with:
directory: core
python: ${{ env.python }}
- name: Test core
run: pytest -v --cov=core --cov-report=xml core/
- uses: tlambert03/setup-qt-libs@v1
- name: Install plugin
timeout-minutes: 10
run: |
/sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1920x1200x24 -ac +extension GLX
pip install pytest-qt PyQt5
pip install -e './plugin[testing]'
- name: Lint plugin
uses: ./.github/lint
with:
directory: plugin
python: ${{ env.python }}
- name: Test plugin
run: pytest -v --cov=plugin --cov-report=xml plugin
- name: Coverage
uses: codecov/codecov-action@v3
deploy:
# this will run when you have tagged a commit, starting with "v*"
# and requires that you have put your twine API key in your
# github secrets (see readme for details)
needs: [test]
runs-on: ubuntu-latest
if: contains(github.ref, 'tags')
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.x"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -U setuptools setuptools_scm wheel twine
- name: Build and publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.TWINE_TOKEN }}
run: |
git tag
python setup.py sdist bdist_wheel
twine upload dist/*
14 changes: 3 additions & 11 deletions core/lls_core/__init__.py
Original file line number Diff line number Diff line change
@@ -1,26 +1,18 @@

__version__ = "0.2.6"


from strenum import StrEnum
from enum import Enum

# Initialize configuration options

#Deskew Direction
from pyclesperanto_prototype._tier8._affine_transform_deskew_3d import DeskewDirection

# Initialize configuration options

#Choice of Deconvolution
class DeconvolutionChoice(Enum):
class DeconvolutionChoice(StrEnum):
cuda_gpu = "cuda_gpu"
opencl_gpu = "opencl_gpu"
cpu = "cpu"

#Choice of File extension to save
class SaveFileType(Enum):
h5 = "h5"
tiff = "tiff"

#CONFIGURE LOGGING using a dictionary (can also be done with yaml file)
import logging.config
LOGGING_CONFIG = {
Expand Down
Loading
Loading