Skip to content

Commit

Permalink
Release v0.4.0. (#24)
Browse files Browse the repository at this point in the history
  • Loading branch information
tobiasraabe authored Oct 7, 2023
1 parent ad5c098 commit 9ae05f5
Show file tree
Hide file tree
Showing 19 changed files with 273 additions and 569 deletions.
27 changes: 9 additions & 18 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,26 +27,17 @@ jobs:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
python-version: ['3.8', '3.9', '3.10', '3.11']
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']

steps:
- uses: actions/checkout@v4
- uses: mamba-org/setup-micromamba@v1
- uses: julia-actions/setup-julia@v1
- uses: actions/setup-python@v4
with:
environment-name: gha-testing
condarc: |
channels:
- nodefaults
- conda-forge
create-args: >-
python=${{ matrix.python-version }}
mamba
tox-conda
cache-environment: true

- name: Install core dependencies.
shell: bash -l {0}
run: mamba install -c conda-forge tox-conda coverage mamba
python-version: ${{ matrix.python-version }}
cache: pip
allow-prereleases: true
- run: pip install tox

# Unit, integration, and end-to-end tests.

Expand All @@ -55,7 +46,7 @@ jobs:
run: tox -e pytest -- -m "unit or (not integration and not end_to_end)" --cov=./ --cov-report=xml -n auto

- name: Upload coverage report for unit tests and doctests.
if: runner.os == 'Linux' && matrix.python-version == '3.8'
if: runner.os == 'Linux' && matrix.python-version == '3.10'
shell: bash -l {0}
run: bash <(curl -s https://codecov.io/bash) -F unit -c

Expand All @@ -64,6 +55,6 @@ jobs:
run: tox -e pytest -- -m end_to_end --cov=./ --cov-report=xml -n auto

- name: Upload coverage reports of end-to-end tests.
if: runner.os == 'Linux' && matrix.python-version == '3.8'
if: runner.os == 'Linux' && matrix.python-version == '3.10'
shell: bash -l {0}
run: bash <(curl -s https://codecov.io/bash) -F end_to_end -c
5 changes: 1 addition & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,13 +64,10 @@ repos:
rev: 'v1.5.1'
hooks:
- id: mypy
args: [
--no-strict-optional,
--ignore-missing-imports,
]
additional_dependencies: [
attrs>=21.3.0,
click,
pytask>=0.4.0,
types-PyYAML,
types-setuptools
]
Expand Down
6 changes: 5 additions & 1 deletion CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ chronological order. Releases follow [semantic versioning](https://semver.org/)
releases are available on [PyPI](https://pypi.org/project/pytask-julia) and
[Anaconda.org](https://anaconda.org/conda-forge/pytask-julia).

## 0.3.0 - 2023-xx-xx
## 0.4.0 - 2023-10-08

- {pull}`24` prepares the release of pytask v0.4.0.

## 0.3.0 - 2023-01-24

- {pull}`16` adds mypy, refurb, and ruff.
- {pull}`18` deprecates INI configurations and aligns the package with pytask v0.3.
Expand Down
33 changes: 16 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,12 @@ task module to the Julia script.

```python
import pytask
from pathlib import Path
from pytask import task


@task(kwargs={"path": Path("out.csv")})
@pytask.mark.julia(script="script.jl")
@pytask.mark.produces("out.csv")
def task_run_jl_script():
pass
```
Expand All @@ -64,11 +66,12 @@ more information.

### Dependencies and Products

Dependencies and products can be added as with a normal pytask task using the
`@pytask.mark.depends_on` and `@pytask.mark.produces` decorators. which is explained in
this
Dependencies and products can be added as usual. Read this
[tutorial](https://pytask-dev.readthedocs.io/en/stable/tutorials/defining_dependencies_products.html).

For example, with the `@pytask.task` decorator as shown before. (The choice of the kwarg
name, here `path`, is arbitrary.)

### Accessing dependencies and products in the script

To access the paths of dependencies and products in the script, pytask-julia stores the
Expand All @@ -82,21 +85,19 @@ path_to_json = ARGS[1] # Contains the path to the .json file.

config = JSON.parse(read(path_to_json, String)) # A dictionary.

config["produces"] # Is the path to the output file "../out.csv".
config["path"] # Is the path to the output file "../out.csv".
```

The `.json` file is stored in the same folder as the task in a `.pytask` directory.

To parse the JSON file, you need to install
[JSON.jl](https://github.com/JuliaIO/JSON.jl).

You can also pass any other information to your script by using the `@pytask.mark.task`
decorator.
You can also pass any other information to your script by using the `@task` decorator.

```python
@pytask.mark.task(kwargs={"number": 1})
@task(kwargs={"path": Path("out.csv"), "number": 1})
@pytask.mark.julia(script="script.jl")
@pytask.mark.produces("out.csv")
def task_run_jl_script():
pass
```
Expand Down Expand Up @@ -141,8 +142,8 @@ You can also define environments for each task which will overwrite any other de
with the `project` keyword argument. Pass a path to the task module.

```python
@task(kwargs={"path": Path("out.csv")})
@pytask.mark.julia(script="script.jl", project=".")
@pytask.mark.produces("out.csv")
def task_run_jl_script():
pass
```
Expand All @@ -152,8 +153,8 @@ def task_run_jl_script():
Command line options can be pass via the `options` keyword argument.

```python
@task(kwargs={"path": Path("out.csv")})
@pytask.mark.julia(script="script.jl", options=["--threads", "2"])
@pytask.mark.produces("out.csv")
def task_run_jl_script():
pass
```
Expand All @@ -171,22 +172,20 @@ produce different outputs.
```python
for i in range(2):

@pytask.mark.task
@task(kwargs={"path": Path(f"out_{i}.csv")})
@pytask.mark.julia(script=f"script_{i}.jl")
@pytask.mark.produces(f"out_{i}.csv")
def task_execute_julia_script():
pass
```

If you want to pass different inputs to the same Julia script, pass these arguments with
the `kwargs` keyword of the `@pytask.mark.task` decorator.
the `kwargs` keyword of the `@task` decorator.

```python
for i in range(2):

@pytask.mark.task(kwargs={"i": i})
@task(kwargs={"path": Path(f"out_{i}.csv"), "i": i})
@pytask.mark.julia(script="script.jl")
@pytask.mark.produces(f"output_{i}.csv")
def task_execute_julia_script():
pass
```
Expand All @@ -200,7 +199,7 @@ path_to_json = ARGS[1] # Contains the path to the .json file.

config = JSON.parse(read(path_to_json, String)) # A dictionary.

config["produces"] # Is the path to the output file "../output_{i}.csv".
config["path"] # Is the path to the output file "../output_{i}.csv".

config["i"] # Is the number.
```
Expand Down
6 changes: 4 additions & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ name: pytask-julia

channels:
- conda-forge
- conda-forge/label/pytask_rc
- conda-forge/label/pytask_parallel_rc
- nodefaults

dependencies:
Expand All @@ -12,8 +14,8 @@ dependencies:

# Package dependencies
- julia
- pytask >=0.3
- pytask-parallel >=0.3
- pytask >=0.4.0
- pytask-parallel >=0.4.0

# Optional package dependencies
- pyyaml
Expand Down
11 changes: 11 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,14 @@ extend-ignore = [

[tool.ruff.pydocstyle]
convention = "numpy"

[tool.pytest.ini_options]
# Do not add src since it messes with the loading of pytask-parallel as a plugin.
testpaths = ["tests"]
markers = [
"wip: Tests that are work-in-progress.",
"unit: Flag for unit tests which target mainly a single function.",
"integration: Flag for integration tests which may comprise of multiple unit tests.",
"end_to_end: Flag for tests that cover the whole program.",
]
norecursedirs = [".idea", ".tox"]
4 changes: 2 additions & 2 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ project_urls =
[options]
packages = find:
install_requires =
pybaum>=0.1.1
pytask>=0.3
pluggy>=1.0.0
pytask>=0.4.0
python_requires = >=3.8
include_package_data = True
package_dir = =src
Expand Down
Loading

0 comments on commit 9ae05f5

Please sign in to comment.