0.9.4 (2024-12-16)
REANA 0.9.4 is a minor update that adds support for using user secrets in Jupyter notebook sessions, adds support for the Compute4PUNCH infrastructure, fixes issues with the HTCondor compute backend job dispatch, and improves the security of the platform.
Please see the REANA 0.9.4 release blog post for more information.
reana 0.9.4 (2024-12-03)
- [Build] helm: add support for Kubernetes 1.30 (#799) (748ca07)
- [Build] helm: add support for Kubernetes 1.31 (#822) (7da51d3)
- [Features] config: add Compute4PUNCH backend (#780) (c2f490b)
- [Features] helm: allow cluster administrator to configure ingress host (#804) (1479730)
- [Features] helm: allow only reana-server to connect to reana-cache (#847) (e1772ff)
- [Features] helm: release check on most-supported Kubernetes version (#848) (1a98b0a)
- [Features] helm: support password-protected rabbitmq (#847) (20a0ea8)
- [Features] helm: support password-protected redis (#847) (be12076)
- [Features] scripts: upgrade to Jupyter SciPy 7.2.2 notebook (#846) (1ca9dea)
- [Bug fixes] helm: allow interactive-session-cleanup job to access RWC (#853) (b9bc602)
- [Bug fixes] reana-dev: correctly handle missing changelog of components (#858) (32549d1), closes #857
- [Continuous integration] python: pin setuptools 70 (#822) (be45c54)
- [Documentation] helm: clarify secrets-related warning in README (#847) (fab5591)
reana-client 0.9.4 (2024-11-29)
- [Build] docker: create
reana-client
container image (#710) (2c99c5d), closes #709 - [Build] python: add support for Python 3.13 (#736) (fd9b944)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#736) (778df03)
- [Continuous integration] actions: pin setuptools 70 (#728) (0a4bcc7)
- [Documentation] cli: fix
open
command documentation typo (#728) (c822dd6)
reana-commons 0.9.9 (2024-11-28)
- [Build] python: add support for Python 3.13 (#480) (5de7605)
- [Features] schema: allow Compute4PUNCH backend options (#445) (0570f4a)
- [Bug fixes] config: remove hard-coded component host name domain (#458) (f2faeaa), closes #457
- [Continuous integration] actions: pin setuptools 70 (#479) (b80bc70)
reana-db 0.9.5 (2024-11-26)
- [Features] cli: add new
migrate-secret-key
command (#240) (efcbe72) - [Continuous integration] actions: pin setuptools 70 (#239) (3202759)
- [Continuous integration] python: test more Python versions (#239) (e0cba7f)
reana-job-controller 0.9.4 (2024-11-29)
- [Build] deps: update reana-auth-vomsproxy to 1.3.0 (#466) (72e9ea1)
- [Build] docker: pin setuptools 70 (#465) (c593d9b)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#477) (9cdd06c)
- [Features] backends: add new Compute4PUNCH backend (#430) (4243252)
- [Bug fixes] config: read secret key from env (#476) (1b5aa98)
- [Bug fixes] config: update reana-auth-vomsproxy to 1.2.1 to fix WLCG IAM (#457) (132868f)
- [Bug fixes] htcondorcern: run provided command in unpacked image (#474) (9cda591), closes #471
- [Bug fixes] htcondorcern: support multiline commands (#474) (eb07aa9), closes #470
- [Bug fixes] kubernetes: avoid privilege escalation in Kubernetes jobs (#476) (389f0ea)
reana-server 0.9.4 (2024-11-29)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#714) (94fbf77)
- [Features] config: make ACCOUNTS_USERINFO_HEADERS customisable (#713) (8c01d51)
- [Features] config: make APP_DEFAULT_SECURE_HEADERS customisable (#713) (1919358)
- [Features] config: make PROXYFIX_CONFIG customisable (#713) (5b6c276)
- [Features] config: support password-protected redis (#713) (a2aad8a)
- [Features] ext: improve error message for db decryption error (#713) (bbab1bf)
- [Bug fixes] config: do not set DEBUG programmatically (#713) (c98cbc1)
- [Bug fixes] config: read secret key from env (#713) (6ee6422)
- [Bug fixes] get_workflow_specification: avoid returning null parameters (#689) (46633d6)
- [Bug fixes] reana-admin: respect service domain when cleaning sessions (#687) (ede882d)
- [Bug fixes] set_workflow_status: publish workflows to submission queue (#691) (6e35bd7), closes #690
- [Bug fixes] start: validate endpoint parameters (#689) (d2d3673)
- [Continuous integration] commitlint: improve checking of merge commits (#689) (69f45fc)
reana-workflow-controller 0.9.4 (2024-11-29)
- [Build] docker: pin setuptools 70 (#601) (be6a388)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#620) (179fa89)
- [Features] config: upgrade to Jupyter SciPy 7.2.2 notebook (#614) (72f0c4c)
- [Features] helm: allow cluster administrator to configure ingress host (#588) (a7c9c85)
- [Features] sessions: expose user secrets in interactive sessions (#591) (784efee)
- [Bug fixes] config: read secret key from env (#615) (7df1279)
- [Bug fixes] manager: avoid privilege escalation in Kubernetes jobs (#615) (24563e5)
- [Bug fixes] manager: pass RabbitMQ connection details to workflow engine (#615) (cf4ee73)
- [Bug fixes] set_workflow_status: validate endpoint arguments (#589) (5945d7f)
reana-workflow-engine-cwl 0.9.4 (2024-11-29)
- [Build] docker: pin setuptools 70 (#287) (3c2cd8a)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#289) (f9d3688)
- [Features] task: allow Compute4PUNCH backend options (#277) (9b2a3d0)
reana-workflow-engine-serial 0.9.4 (2024-11-29)
- [Build] docker: pin setuptools 70 (#216) (f94d003)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#218) (430fd04)
- [Features] tasks: allow Compute4PUNCH backend options (#210) (a6313f2)
reana-workflow-engine-snakemake 0.9.4 (2024-11-29)
- [Build] docker: fix XRootD repository location (#95) (69fea32)
- [Build] docker: pin setuptools 70 (#102) (b27c9cf)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#104) (fb9efc8)
- [Features] executor: allow Compute4PUNCH backend options (#97) (4b00c52)
- [Bug fixes] executor: override default resources to remove mem/disk (#91) (572a83f), closes #90
reana-workflow-engine-yadage 0.9.5 (2024-11-29)
- [Build] docker: pin setuptools 70 (#274) (bc505d8)
- [Build] python: bump shared REANA packages as of 2024-11-28 (#276) (5911143)
- [Features] externalbackend: allow Compute4PUNCH backend options (#269) (1ce8e6a)
0.9.3 (2024-03-13)
REANA 0.9.3 is a minor update that upgrades Snakemake workflow engine to version 7, improves job submission performance for massively-parallel workflows, improves the clean-up processing for stopped and failed workflows, and brings other minor improvements and bug fixes.
Please see the REANA 0.9.3 release blog post for more information.
reana 0.9.3 (2024-03-13)
- [Build] helm: add support for Kubernetes 1.29 (#775) (ae90500)
- [Features] helm: add value to customise env vars of job controller (#781) (634691f)
- [Features] helm: add value to customise PostgreSQL docker image (#774) (07a191f), closes #773
- [Features] helm: add value to customise URL of privacy notice (#778) (650ddbd)
- [Features] helm: add values to customise env vars of workflow engines (#781) (35ee032)
- [Features] helm: use PostgreSQL 14.10 in local dev deployment (#774) (43ead8a), closes #744
- [Features] reana-dev: add
git-aggregate-changelog
(#789) (6210b11) - [Features] reana-dev: add date to commits bumping dependencies (#787) (a4cb84c)
- [Bug fixes] reana-dev: add PR number in commits bumping shared modules (#783) (57c6755)
- [Bug fixes] reana-dev: create commits that conform to conventional style (#777) (86d0133)
- [Bug fixes] reana-dev: delete extra files with git-submodule --update (#764) (e5680ce)
- [Bug fixes] reana-dev: detect number of already open PR in commit suffix (#783) (c533c34)
- [Bug fixes] reana-dev: update container image labels when releasing (#765) (fe6bc2c)
- [Code refactoring] docs: move from reST to Markdown (#776) (79aedb9)
- [Code style] black: format with black v24 (#772) (311e157)
- [Continuous integration] commitlint: addition of commit message linter (#767) (be77666)
- [Continuous integration] commitlint: allow release commit style (#785) (a6f95ac)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#771) (2c34634)
- [Continuous integration] release-please: initial configuration (#767) (bb45539)
- [Continuous integration] release-please: update version in Helm Chart (#770) (09c9210)
- [Continuous integration] shellcheck: fix exit code propagation (#771) (035d51c)
- [Documentation] authors: complete list of contributors (#779) (123eae8)
- [Documentation] chat: remove Gitter (#782) (aba8ac2)
reana-client 0.9.3 (2024-03-13)
- [Build] appimage: upgrade to Python 3.8.18 (#704) (783c17a)
- [Bug fixes] status: display correct duration of stopped workflows (#701) (b53def8), closes #699
- [Code refactoring] docs: move from reST to Markdown (#703) (c9c4d53)
- [Code style] black: format with black v24 (#702) (02dc830)
- [Test suite] snakemake: allow running Snakemake 7 tests on Python 3.11+ (#700) (8ad7ff1), closes #655
- [Continuous integration] commitlint: addition of commit message linter (#695) (2de7d61)
- [Continuous integration] commitlint: allow release commit style (#708) (f552752)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#698) (fa5b7c7)
- [Continuous integration] pytest: install
tests
package variant instead ofall
(#703) (fe0b00a) - [Continuous integration] release-please: initial configuration (#695) (5b278f1)
- [Continuous integration] shellcheck: fix exit code propagation (#698) (fe696ea)
- [Documentation] authors: complete list of contributors (#705) (875997c)
reana-commons 0.9.8 (2024-03-01)
- [Build] python: change extra names to comply with PEP 685 (#446) (9dad6da)
- [Build] python: require smart-open<7 for Python 3.6 (#446) (17fd581)
- [Build] python: restore snakemake
reports
extra (#446) (904178f) - [Continuous integration] commitlint: allow release commit style (#447) (1208ccf)
reana-commons 0.9.7 (2024-02-20)
- [Build] snakemake: require pulp<2.8.0 (#444) (5daa109)
- [Documentation] authors: complete list of contributors (#442) (4a74c10)
reana-commons 0.9.6 (2024-02-13)
- [Features] config: allow customisation of runtime group name (#440) (5cec305)
- [Features] snakemake: upgrade to Snakemake 7.32.4 (#435) (20ae9ce)
- [Bug fixes] cache: handle deleted files when calculating access times (#437) (698900f)
- [Code refactoring] docs: move from reST to Markdown (#441) (36ce4e0)
- [Continuous integration] commitlint: addition of commit message linter (#432) (a67906f)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#438) (d3035dc)
- [Continuous integration] release-please: initial configuration (#432) (687f2f4)
- [Continuous integration] shellcheck: check all shell scripts recursively (#436) (709a685)
- [Continuous integration] shellcheck: fix exit code propagation (#438) (85d9a2a)
- Fixes installation by pinning
bravado-core
to versions lower than 6.1.1.
reana-db 0.9.4 (2024-03-01)
- [Code refactoring] docs: move from reST to Markdown (#225) (b48eb55)
- [Code style] black: format with black v24 (#224) (cc60522)
- [Continuous integration] commitlint: addition of commit message linter (#218) (ee0f7e5)
- [Continuous integration] commitlint: allow release commit style (#229) (adf15d7)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#223) (3d513f6)
- [Continuous integration] pytest: move to PostgreSQL 14.10 (#226) (4dac889)
- [Continuous integration] release-please: initial configuration (#218) (7c616d6)
- [Continuous integration] shellcheck: fix exit code propagation (#223) (b62ee1e)
- [Documentation] authors: complete list of contributors (#227) (3fbcf65)
reana-job-controller 0.9.3 (2024-03-04)
- [Build] certificates: update expired CERN Grid CA certificate (#440) (8d6539a), closes #439
- [Build] docker: non-editable submodules in "latest" mode (#416) (3bdda63)
- [Build] python: bump all required packages as of 2024-03-04 (#442) (de119eb)
- [Build] python: bump shared REANA packages as of 2024-03-04 (#442) (fc77628)
- [Features] shutdown: stop all running jobs before stopping workflow (#423) (866675b)
- [Bug fixes] database: limit the number of open database connections (#437) (980f749)
- [Performance improvements] cache: avoid caching jobs when the cache is disabled (#435) (553468f), closes #422
- [Code refactoring] db: set job status also in the main database (#423) (9d6fc99)
- [Code refactoring] docs: move from reST to Markdown (#428) (4732884)
- [Code refactoring] monitor: centralise logs and status updates (#423) (3685b01)
- [Code refactoring] monitor: move fetching of logs to job-manager (#423) (1fc117e)
- [Code style] black: format with black v24 (#426) (8a2757e)
- [Continuous integration] commitlint: addition of commit message linter (#417) (f547d3b)
- [Continuous integration] commitlint: allow release commit style (#443) (0fc9794)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#425) (35bc1c5)
- [Continuous integration] pytest: move to PostgreSQL 14.10 (#429) (42622fa)
- [Continuous integration] release-please: initial configuration (#417) (fca6f74)
- [Continuous integration] release-please: update version in Dockerfile/OpenAPI specs (#421) (e6742f2)
- [Continuous integration] shellcheck: fix exit code propagation (#425) (8e74a85)
- [Documentation] authors: complete list of contributors (#434) (b9f8364)
reana-message-broker 0.9.3 (2024-03-01)
- [Bug fixes] startup: handle signals for graceful shutdown (#59) (abb8969)
- [Code refactoring] docs: move from reST to Markdown (#65) (e5bd869)
- [Continuous integration] commitlint: addition of commit message linter (#60) (a9ee4bb)
- [Continuous integration] commitlint: allow release commit style (#67) (600aa01)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#64) (8283064)
- [Continuous integration] release-please: initial configuration (#60) (02b3595)
- [Continuous integration] release-please: update version in Dockerfile (#63) (548f3e1)
- [Continuous integration] shellcheck: fix exit code propagation (#64) (8a2a7fc)
- [Documentation] authors: complete list of contributors (#66) (56fbc8c)
reana-server 0.9.3 (2024-03-04)
- [Build] deps: pin invenio-userprofiles to 1.2.4 (#665) (d6cb168)
- [Build] docker: non-editable submodules in "latest" mode (#656) (d16fefb)
- [Build] python: bump all required packages as of 2024-03-04 (#674) (f40b82f)
- [Build] python: bump shared modules (#676) (47ad3ca)
- [Build] python: bump shared REANA packages as of 2024-03-04 (#674) (aa18394)
- [Code refactoring] docs: move from reST to Markdown (#671) (b6d1799)
- [Code style] black: format with black v24 (#670) (6d2b898)
- [Continuous integration] commitlint: addition of commit message linter (#665) (2b43ecc)
- [Continuous integration] commitlint: allow release commit style (#675) (e0299ef)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#669) (87c6145)
- [Continuous integration] pytest: move to PostgreSQL 14.10 (#672) (e888ddd)
- [Continuous integration] release-please: initial configuration (#665) (1d5e7c5)
- [Continuous integration] release-please: update version in Dockerfile/OpenAPI specs (#668) (3b3dc41)
- [Continuous integration] shellcheck: fix exit code propagation (#669) (d7eac6b)
- [Documentation] authors: complete list of contributors (#673) (71b3f38)
reana-ui 0.9.4 (2024-03-04)
- [Build] package: require jsroot<7.6.0 (#399) (d53b290)
- [Build] package: update yarn.lock (#399) (10e41b1)
- [Features] footer: link privacy notice to configured URL (#393) (f0edde6), closes #392
- [Bug fixes] launcher: remove dollar sign in generated Markdown (#389) (8ad4afd)
- [Bug fixes] progress: update failed workflows duration using finish time (#387) (809fdc5), closes #386
- [Bug fixes] router: show 404 page for invalid URLs (#382) (c18e81d), closes #379
- [Code refactoring] docs: move from reST to Markdown (#391) (8d58277)
- [Continuous integration] commitlint: addition of commit message linter (#380) (1c9ec74)
- [Continuous integration] commitlint: allow release commit style (#400) (426a2b0)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#390) (e938f60)
- [Continuous integration] release-please: initial configuration (#380) (db2e82b)
- [Continuous integration] release-please: switch to
simple
release strategy (#383) (2c64085) - [Continuous integration] release-please: update version in package.json and Dockerfile (#385) (5d232af)
- [Continuous integration] shellcheck: exclude node_modules from the analyzed paths (#387) (8913e4d)
- [Continuous integration] shellcheck: fix exit code propagation (#390) (7b5f29e)
- [Documentation] authors: complete list of contributors (#396) (814d68e)
reana-workflow-controller 0.9.3 (2024-03-04)
- [Build] docker: non-editable submodules in "latest" mode (#551) (af74d0b)
- [Build] python: bump all required packages as of 2024-03-04 (#574) (1373f4c)
- [Build] python: bump shared REANA packages as of 2024-03-04 (#574) (e31d903)
- [Features] manager: call shutdown endpoint before workflow stop (#559) (719fa37)
- [Features] manager: increase termination period of run-batch pods (#572) (f05096a)
- [Features] manager: pass custom env variables to job controller (#571) (646f071)
- [Features] manager: pass custom env variables to workflow engines (#571) (cb9369b)
- [Bug fixes] manager: graceful shutdown of job-controller (#559) (817b019)
- [Bug fixes] manager: use valid group name when calling
groupadd
(#566) (73a9929), closes #561 - [Bug fixes] stop: store engine logs of stopped workflow (#563) (199c163), closes #560
- [Code refactoring] consumer: do not update status of jobs (#559) (5992034)
- [Code refactoring] docs: move from reST to Markdown (#567) (4fbdb74)
- [Code style] black: format with black v24 (#564) (2329437)
- [Continuous integration] commitlint: addition of commit message linter (#555) (b9df20a)
- [Continuous integration] commitlint: allow release commit style (#575) (b013d49)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#562) (4b8f539)
- [Continuous integration] pytest: move to PostgreSQL 14.10 (#568) (9b6bfa0)
- [Continuous integration] release-please: initial configuration (#555) (672083d)
- [Continuous integration] release-please: update version in Dockerfile/OpenAPI specs (#558) (4be8086)
- [Continuous integration] shellcheck: fix exit code propagation (#562) (c5d4982)
- [Documentation] authors: complete list of contributors (#570) (08ab9a3)
reana-workflow-engine-cwl 0.9.3 (2024-03-04)
- [Build] docker: install correct extras of reana-commons submodule (#261) (21957fe)
- [Build] docker: non-editable submodules in "latest" mode (#255) (a6acc88)
- [Build] python: bump all required packages as of 2024-03-04 (#267) (ed6a846)
- [Build] python: bump shared REANA packages as of 2024-03-04 (#267) (47155ef)
- [Bug fixes] progress: handle stopped jobs (#260) (bc36cb7)
- [Code refactoring] docs: move from reST to Markdown (#263) (3cf272f)
- [Continuous integration] commitlint: addition of commit message linter (#256) (021854e)
- [Continuous integration] commitlint: allow release commit style (#268) (ed7ad11)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#262) (9a45817)
- [Continuous integration] release-please: initial configuration (#256) (bcd87d1)
- [Continuous integration] release-please: update version in Dockerfile (#259) (0961257)
- [Continuous integration] shellcheck: fix exit code propagation (#262) (6568b9b)
- [Documentation] authors: complete list of contributors (#266) (2960cd9)
- [Documentation] conformance-tests: update CWL conformance test badges (#264) (45afa2e)
reana-workflow-engine-serial 0.9.3 (2024-03-04)
- [Build] docker: install correct extras of reana-commons submodule (#196) (b23f4df)
- [Build] docker: non-editable submodules in "latest" mode (#190) (03a15cf)
- [Build] python: bump all required packages as of 2024-03-04 (#200) (ffc8aec)
- [Build] python: bump shared REANA packages as of 2024-03-04 (#200) (47c26cc)
- [Bug fixes] progress: handle stopped jobs (#195) (a232a76)
- [Code refactoring] docs: move from reST to Markdown (#198) (7507d12)
- [Continuous integration] commitlint: addition of commit message linter (#191) (b7a6ef1)
- [Continuous integration] commitlint: allow release commit style (#201) (b50b6d0)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#197) (1813ac3)
- [Continuous integration] release-please: initial configuration (#191) (d40a675)
- [Continuous integration] release-please: update version in Dockerfile (#194) (52c34ec)
- [Continuous integration] shellcheck: fix exit code propagation (#197) (5565b29)
- [Documentation] authors: complete list of contributors (#199) (e9b25b6)
reana-workflow-engine-snakemake 0.9.3 (2024-03-04)
- [Build] docker: install correct extras of reana-commons submodule (#79) (fd9b88a)
- [Build] docker: non-editable submodules in "latest" mode (#73) (c3595c2)
- [Build] python: bump all required packages as of 2024-03-04 (#85) (66e81e2)
- [Build] python: bump shared REANA packages as of 2024-03-04 (#85) (d07f91f)
- [Features] config: get max number of parallel jobs from env vars (#84) (69cfad4)
- [Features] executor: upgrade to Snakemake v7.32.4 (#81) (4a3f359)
- [Bug fixes] progress: handle stopped jobs (#78) (4829d80)
- [Code refactoring] docs: move from reST to Markdown (#82) (31de94f)
- [Continuous integration] commitlint: addition of commit message linter (#74) (145b7e7)
- [Continuous integration] commitlint: allow release commit style (#86) (fd032db)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#80) (b677913)
- [Continuous integration] release-please: initial configuration (#74) (9b16bd0)
- [Continuous integration] release-please: update version in Dockerfile (#77) (3c35a67)
- [Continuous integration] shellcheck: fix exit code propagation (#80) (ad15c0d)
- [Documentation] authors: complete list of contributors (#83) (4782678)
reana-workflow-engine-yadage 0.9.4 (2024-03-04)
- [Build] docker: install correct extras of reana-commons submodule (#256) (8b4caf0)
- [Build] docker: non-editable submodules in "latest" mode (#249) (a57716a)
- [Build] python: bump all required packages as of 2024-03-04 (#261) (2a02e19)
- [Build] python: bump shared REANA packages as of 2024-03-04 (#261) (493aee1)
- [Bug fixes] progress: correctly handle running and stopped jobs (#258) (56ef6a4)
- [Code refactoring] docs: move from reST to Markdown (#259) (37f05e6)
- [Continuous integration] commitlint: addition of commit message linter (#251) (f180e21)
- [Continuous integration] commitlint: allow release commit style (#262) (1b8b6b8)
- [Continuous integration] commitlint: check for the presence of concrete PR number (#257) (9ddb488)
- [Continuous integration] release-please: initial configuration (#251) (dc4fa7a)
- [Continuous integration] release-please: update version in Dockerfile (#254) (8f18751)
- [Continuous integration] shellcheck: fix exit code propagation (#257) (8831d9e)
- [Documentation] authors: complete list of contributors (#260) (68f97a0)
-
Users:
- Adds web interface form allowing to generate launcher URL for any user-provided public analysis, as well as the Markdown snippet for the corresponding Launch-on-REANA badge.
- Adds web interface option to delete all the runs of a workflow.
- Changes the Launch-on-REANA web interface page to improve how workflow parameters are shown by displaying them inside a table.
- Changes CVMFS support to allow users to automatically mount any available repository.
- Changes the REANA specification schema to use the
draft-07
version of the JSON Schema specification. - Changes
reana-client validate
command to show detailed errors when the specification file is not a valid YAML file. - Changes validation of REANA specification to make the
environment
property mandatory for the steps of serial workflows. - Changes validation of REANA specification to raise a warning for unexpected properties for the steps of serial workflows.
- Changes validation of REANA specification to report improved validation warnings which also indicate where unexpected properties are located in the file.
- Changes workflow restarts to allow for more than nine restarts of the same workflows.
- Changes workflow scheduler logging behaviour to also report the main reason behind scheduling errors to the users.
- Fixes
reana-client list
command to accept case-insensitive column names when sorting the returned workflow runs via the--sort
option. - Fixes
reana-client run
wrapper command for workflows that do not containinputs
clause in their specification. - Fixes
reana-client
'screate_workflow_from_json
API function to always load and send the workflow specification to the server. - Fixes creation of image thumbnails for output files in Snakemake HTML execution reports.
-
Administrators:
- Changes several database index definitions in order to improve performance of most common database queries.
- Changes the names of database table, column, index and key constraints in order to follow the SQLAlchemy upstream naming conventions everywhere.
- Changes the
Workflow
table to replace therun_number
column with two new columnsrun_number_major
andrun_number_minor
in order to allow for more than nine restarts of user workflows. - Changes CVMFS support to allow users to automatically mount any available repository, thanks to CVMFS CSI v2.
- Fixes the mounting of CVMFS volumes for the REANA deployments that use non-default Kubernetes namespace.
- Fixes container image building of cluster components for the arm64 architecture.
- Fixes job monitoring in cases when job creation fails, for example when it is not possible to successfully mount volumes.
- Fixes job status consumer exception while attempting to fetch workflow engine logs for workflows that could not have been successfully scheduled.
- Fixes the creation of Kubernetes jobs by retrying in case of error and by correctly handling the error after reaching the retry limit.
-
Developers:
- Adds automated multi-platform container image building of cluster components for amd64 and arm64 architectures.
- Adds new
--image-name
option to thereana-dev docker-push
command to customise the name of the docker image to publish. - Adds new
--platform
option to thereana-dev docker-build
andreana-dev release-docker
commands to build and publish multi-platform images. - Adds new
--registry
option to thereana-dev docker-push
andreana-dev release-docker
commands to specify the registry where the built docker images should be pushed to. - Adds new
--tags-only
option to thereana-dev release-docker
command to only print the final docker image names, without pushing the images to the registry. - Adds new
reana-dev git-create-release-branch
command to create a new Git branch for a new release. - Adds new
reana-dev git-upgrade-requirements
command to upgrade therequirements.txt
file before a new release. - Changes
reana-dev git-fork
andreana-dev git-create-pr
to use thegh
CLI client instead ofhub
. - Changes
reana-dev python-run-tests
command to allow excluding certain Python components. - Changes
reana-dev python-run-tests
command to allow execution of selected pytests only by passing overPYTEST_ADDOPTS
environment variable. - Changes validation of REANA specification to expose functions for loading workflow input parameters and workflow specifications.
- Changes version of
reana-ui
Node.js Docker image from 16 to 18. - Changes the workflow deletion endpoint to return a different and more appropriate message when deleting all the runs of a workflow.
- Changes the workflow list endpoint on how pagination is performed in order to avoid counting twice the total number of records.
- Fixes
reana-dev python-run-tests
command to create Python-3.8 based virtual environments to use the same version as container images.
-
Users:
- Adds support for Python 3.12.
- Adds support for previewing PDF files present in a workflow's workspace.
- Adds support for previewing ROOT files present in a workflow's workspace.
- Adds support for signing-in with a custom third-party Keycloak instance.
- Adds a new menu item to the workflow actions popup to allow stopping running workflows.
- Adds
prune
command to delete all intermediate files of a given workflow. Use with care. - Changes the deletion of a workflow to automatically delete an open interactive session attached to its workspace.
- Changes the workflow deletion message to clarify that attached interactive sessions are also closed when a workflow is deleted.
- Changes the workflow progress bar to always display it as completed for finished workflows.
- Changes the interactive session notification to also report that the session will be closed after a specified number of days of inactivity.
- Changes the workflow-details page to make it possible to scroll through the list of workflow steps in the job logs section.
- Changes the workflow-details page to not automatically refresh the selected job when viewing the related logs, but keeping the user-selected one active.
- Changes the page titles to conform to the same sentence case style.
- Changes the launcher page to show warnings when validating the REANA specification file of the workflow to be launched.
- Changes
open
command to inform user about the auto-closure of interactive sessions after a certain inactivity timeout. - Changes
validate
command to display non-critical validation warnings when checking the REANA specification file. - Changes Rucio authentication helper to allow users to override the Rucio server and authentication hosts independently of VO name.
- Fixes job status inconsistency when stopping a workflow by setting the job statuses to
stopped
for any running jobs. - Fixes calculation of workflow runtime durations for stopped workflows.
- Fixes
list
command to correctly list workflows when sorting them by their run number or by the size of their workspace. - Fixes
du
command help message typo. - Fixes
validation --environments
command to correctly handle fully-qualified image names. - Fixes deletion of failed jobs not being performed when Kerberos is enabled.
- Fixes job monitoring to consider OOM-killed jobs as failed.
- Fixes detection of default Rucio server and authentication host for ATLAS VO.
- Fixes the reported total number of jobs for restarted workflows by excluding cached jobs that were simply reused from previous runs in the workspace and not really executed by Snakemake.
- Fixes an issue where Snakemake workflows could get stuck waiting for already-finished jobs.
-
Administrators:
- Adds support for Kubernetes clusters 1.26, 1.27, 1.28.
- Adds new configuration option
components.reana_ui.launcher_examples
to customise the demo examples that are shown in the launch page in the REANA UI. - Adds new configuration option
interactive_sessions.maximum_inactivity_period
to set a limit in days for the maximum inactivity period of interactive sessions after which they will be closed. - Adds new configuration option
interactive_sessions.cronjob_schedule
to set how often interactive session cleanup should be performed. - Adds new configuration option
ingress.extra
to define extra Ingress resources, in order to support redirecting HTTP requests to HTTPS with traefik v2 version. - Adds new configuration option
ingress.tls.hosts
to define hosts that are present in the TLS certificate, in order to support cert-manager's automatic creation of certificates. - Adds new configuration option
notifications.email_config.smtp_ssl
to use SSL when connecting to the SMTP email server. - Adds new configuration option
notifications.email_config.smtp_starttls
to use the STARTTLS command to enable encryption after connecting to the SMTP email server. - Adds new configuration option
components.reana_ui.file_preview_size_limit
to set the maximum file size that can be previewed in the web interface. - Adds new configuration options
login
andsecrets.login
for configuring Keycloak SSO login with third-party authentication services. - Adds new
interactive-session-cleanup
command that can be used by REANA administrators to close interactive sessions that are inactive for more than the specified number of days. - Adds progress meter to the logs of the periodic quota updater.
- Adds the content of the
secrets.gitlab.REANA_GITLAB_HOST
configuration option to the list of GitLab instances from which it is possible to launch a workflow. - Changes uWSGI configuration to increase buffer size, add vacuum option, etc.
- Changes CPU and disk quota calculations to improve the performance of periodic quota updater.
- Changes the system status report to simplify and clarify the disk usage summary.
- Changes
check-workflows
command to also check the presence of workspaces on the shared volume. - Changes
check-workflows
command to not show in-sync runs by default. If needed, they can be shown using the new--show-all
option. - Changes
reana-admin
command options to require the passing of--admin-access-token
argument more globally. - Changes the k8s specification for interactive session pods to include labels for improved subset selection of objects.
- Changes the k8s specification for interactive session ingress resource to include annotations.
- Changes nginx configuration to save bandwidth by serving gzip-compressed static files.
- Changes HTCondor to version 9.0.17 (LTS).
- Fixes uWSGI memory consumption on systems with very high allowed number of open files.
- Fixes cronjob failures due to database connection issues when REANA is deployed with non-default namespace or prefix.
- Fixes
ingress.enabled
option to correctly enable or disable the creation of Ingresses. - Fixes graceful shutdown for reana-server and reana-workflow-controller.
- Fixes the workflow priority calculation to avoid workflows stuck in the
queued
status when the number of allowed concurrent workflow is set to zero. - Fixes GitLab integration to automatically redirect the user to the correct URL when the access request is accepted.
- Fixes authentication flow to correctly deny access to past revoked tokens in case the same user has also other new active tokens.
- Fixes email templates to show the correct
kubectl
commands when REANA is deployed inside a non-default namespace or with a custom component name prefix. - Fixes email sender for system emails to
notifications.email_config.sender
Helm value. - Fixes email receiver for token request emails to use
notifications.email_config.receiver
Helm value. - Fixes
quota-set-default-limits
command to propagate default quota limits to all users without custom quota limit values. - Fixes job status consumer to correctly rollback the database transaction when an error occurs.
- Fixes intermittent Slurm connection issues by DNS-resolving the Slurm head node IPv4 address before establishing connections.
- Fixes Slurm command generation issues when using fully-qualified image names.
- Fixes high memory usage of RabbitMQ by limiting the maximum number of open file descriptors.
- Removes support for Kubernetes version prior to 1.21.
-
Developers:
- Adds new
prune_workspace
endpoint to allow users to delete all the files of a workflow, specifying whether to also delete the inputs and/or the outputs. - Adds the timestamp of when the workflow was stopped (
run_stopped_at
) to the workflow list and the workflow status endpoints. - Adds unique error messages to Kubernetes job monitor to more easily identify source of problems.
- Adds new
--parallel
option todocker-build
,cluster-build
andrun-ci
to build multiple docker images in parallel. - Changes
launch
endpoint to also include the warnings of the validation of the workflow specification. - Changes OpenAPI specification of the
info
endpoint to return the maximum inactivity time before automatic closure of interactive sessions. - Changes
apispec
dependency version in order to be compatible withPyYAML
v6. - Changes Paramiko to version 3.0.0.
- Changes remote storage file support for Snakemake workflows to use XRootD 5.6.0.
- Fixes
cluster-deploy
,cluster-undeploy
andclient-setup-environment
commands when using custom instance name or kubernetes namespace. - Fixes the
git-tag
command to display the component name. - Fixes container image names to be Podman-compatible.
- Fixes location of HTCondor build dependencies.
- Adds new
-
Users:
- Adds support for Rucio authentication for workflow jobs.
- Adds support for Kerberos authentication for workflow orchestration.
- Adds support for specifying
slurm_partition
andslurm_time
for Slurm compute backend jobs. - Adds support for XRootD remote file locations in Snakemake workflow specification definitions.
- Adds support for Python 3.11.
- Adds Launch on REANA page allowing the submission of workflows via badge-clicking.
- Adds notifications to inform users when critical levels of quota usage is reached.
- Adds 404 Not Found error page.
- Adds tab titles to all the pages.
- Adds the
REANA_WORKSPACE
environment variable to jupyter notebooks and terminals. - Adds option to sort workflows by most disk and cpu quota usage to the workflow list endpoint.
- Adds support for specifying and listing workspace file retention rules.
- Adds support for
.gitignore
and.reanaignore
to specify files that should not be uploaded to REANA. - Adds
retention-rules-list
command to list the retention rules of a workflow. - Changes REANA specification to allow specifying
retention_days
for the workflow. - Changes default Slurm partition to
inf-short
. - Changes GitLab integration to also retrieve user's projects that are in groups and subgroups.
- Changes the workflow-details page to show the logs of the workflow engine.
- Changes the workflow-details page to show file sizes in a human-readable format.
- Changes the workflow-details page to show the workspace's retention rules.
- Changes the workflow-details page to show the duration of the workflow's jobs.
- Changes the workflow-details page to display a label of the workflow launcher URL remote origin.
- Changes the workflow-details page to periodically refresh the content of the page.
- Changes the workflow-details page to refresh after the deletion of a workflow.
- Changes the workflow-list page to add a way to hide deleted workflows.
- Changes the workflow-list page to add new workflows sorting options by most used disk and cpu quota.
- Changes the deletion of a workflow to always clean up the workspace and to update the user disk quota usage.
- Changes the CWD of jupyter's terminals to the directory of the workflow's workspace.
- Changes percentage ranges used to calculate the health status of user resource quota usage.
- Changes
create
andrestart
commands to always upload the REANA specification file. - Changes
delete
command to always delete the workflow's workspace. - Changes
delete_workflow
Python API function to always delete the workflow's workspace. - Changes
download
command to add the possibility to write files to the standard output via-o -
option. - Changes
list
command to hide deleted workflows by default. - Changes
list
command to allow displaying deleted workflows via--all
and--show-deleted-runs
options. - Changes
list
andstatus
commands to allow displaying the duration of workflows with the--include-duration
option. - Changes
mv
command to allow moving files while a workflow is running. - Changes
upload
command to prevent uploading symlinks. - Changes
validation --environment
command to use Docker registry v2 APIs to check that a Docker image exists in DockerHub. - Fixes
list
command to highlight the workflow specified inREANA_WORKON
correctly. - Fixes
secrets-delete
command error message when deleting non existing secrets. - Fixes
start
command to report failed workflows as errors. - Fixes
start
andrun
commands to correctly follow the execution of the workflow until termination. - Fixes
status
command to respect output format provided by the--format
option. - Fixes
upload
command to report when input directories are listed under thefiles
section in the REANA specification file and vice versa. - Fixes
validate --environment
command to detect illegal whitespace characters in Docker image names. - Fixes Kerberos authentication for long-running workflows by renewing the Kerberos ticket periodically.
- Fixes status reporting for failed CWL and Snakemake jobs that were incorrectly considered successful.
- Fixes redirection chain for non-signed-in CERN SSO users to access the desired target page after sign-in.
- Fixes the ordering by size of the files showed in the
Workspace
tab of the workflow-details page. - Fixes CERN OIDC authentication to possibly allow eduGAIN and social login users.
- Fixes wrong numbering of restarted workflows by limiting the number of times a workflow can be restarted to nine.
-
Administrators:
- Adds new configuration environment variable
reana_server.environment.REANA_SCHEDULER_REQUEUE_COUNT
to set workflow requeue count in case of scheduling errors or busy cluster situations. - Adds "infinity" option to
REANA_SCHEDULER_REQUEUE_COUNT
to disable requeue count. - Adds support for Kubernetes clusters 1.22, 1.23, 1.24, 1.25.
- Adds new configuration option
workspaces.retention_rules.maximum_period
to set a default period for workspace retention rules. - Adds new configuration option
workspaces.retention_rules.cronjob_schedule
to set how often pending retention rules should be applied. - Adds configuration environment variable
reana_server.environment.REANA_RATELIMIT_SLOW
to limit API requests to some protected endpoints e.g launch workflow. - Adds configuration environment variable
reana_server.environment.REANA_WORKFLOW_SCHEDULING_READINESS_CHECK_LEVEL
to define checks that are performed to assess whether the cluster is ready to start new workflows. - Adds new configuration option
ingress.tls.self_signed_cert
to enable the generation of a self-signed TLS certificate. - Adds new configuration option
ingress.tls.secret_name
to specify the name of the Kubernetes secret containing the TLS certificate to be used. - Adds support for configuring an additional volume to be used by the database and the message broker.
- Adds new configuration option
maintenance.enabled
to scale down the cluster for maintenance. - Adds support for Unicode characters inside email body.
- Adds
queue-consume
command that can be used by REANA administrators to remove specific messages from the queue. - Adds
retention-rules-apply
command that can be used by REANA administrators to apply pending retention rules. - Adds
retention-rules-extend
command that can be used by REANA administrators to extend the duration of active retentions rules. - Adds
check-workflows
command that can be used by REANA administrators to check for out-of-sync workflows and interactive sessions. - Changes configuration option
quota.workflow_termination_update_policy
to deactivate workflow termination accounting by default. - Changes Helm template to use PostgreSQL 12.13 version.
- Changes the base image for most of the components to Ubuntu 20.04 LTS and reduces final Docker image size by removing build-time dependencies.
- Changes
reana-auth-vomsproxy
sidecar to the latest stable version to support client-side proxy file generation technique and ESCAPE VOMS. - Changes OAuth configuration to enable the new CERN SSO.
- Changes job status consumer to improve logging for not-alive workflows.
- Changes the deployment of interactive sessions to improve security by not automounting the Kubernetes service account token.
- Changes the deployment of job-controller to avoid unnecessarily mounting the database's directory.
- Changes the announcements to support limited HTML markup.
- Changes REANA specification loading functionality to allow specifying different working directories.
- Changes global setting of maximum number of parallel jobs to 300 for Snakemake workflow engine.
- Fixes job status consumer by discarding invalid job IDs.
- Fixes GitLab integration error reporting in case user exceeds CPU or Disk quota usage limits.
- Fixes issue when irregular number formats are passed to
REANA_SCHEDULER_REQUEUE_COUNT
configuration environment variable. - Fixes quota updater to reduce memory usage.
- Fixes conversion of possibly-negative resource usage values to human-readable formats.
- Fixes disk quota updater to prevent setting negative disk quota usage values.
- Removes support for Kubernetes version prior to 1.19.
- Adds new configuration environment variable
-
Developers:
- Adds OpenAPI specification support for
launch
endpoint that allows running workflows from remote sources. - Adds OpenAPI specification support for
get_workflow_retention_rules
endpoint that allows to retrieve the workspace file retention rules of a workflow. - Adds the remote origin of workflows submitted via Launch-on-REANA (
launcher_url
) to the workflow list endpoint. - Adds common utility functions for managing workspace files to
reana-commons
. - Changes default consumer prefetch count to handle 10 messages instead of 200 in order to reduce the probability of 406 PRECONDITION errors on message acknowledgement.
- Changes
git-upgrade-shared-modules
to generate the correct upper-bound insetup.py
. - Changes REANA specification loading and validation functionalities by porting some of the logic to
reana-commons
. - Changes OpenAPI specification to include missing response schema elements.
- Changes the Kubernetes Python client to use the
networking/v1
API. - Changes the deployment of interactive sessions to use
networking/v1
Kubernetes API. - Changes to Flask v2.
- Changes
/api/info
endpoint to also include the kubernetes maximum memory limit, the kubernetes default memory limit and the maximum workspace retention period. - Changes
start_workflow
endpoint to validate the REANA specification of the workflow. - Changes
create_workflow
endpoint to populate workspace retention rules for the workflow. - Changes
start_workflow
endpoint to disallow restarting a workflow when retention rules are pending. - Changes API rate limiter error messages to be more verbose.
- Changes workflow scheduler to allow defining the checks needed to assess whether the cluster can start new workflows.
- Changes workflow list endpoint to add the possibility to filter by workflow ID.
- Changes the
move_files
endpoint to allow moving files while a workflow is running. - Changes the k8s specification of interactive sessions' pods to remove the environment variables used for service discovery.
- Changes GitLab integration to use
reana
as pipeline name instead ofdefault
when setting status of a commit. - Changes the loading of Snakemake specifications to preserve the current working directory.
- Changes the Invenio dependencies to the latest versions.
- Fixes the submission of jobs by stripping potential leading and trailing whitespaces in Docker image names.
- Fixes
fetchWorkflow
action to fetch a specific workflow instead of the entire user workflow list. (reana-ui) - Fixes the download of files by changing the default MIME type to
application/octet-stream
. - Fixes the workflow list endpoint to correctly parse the boolean parameters
include_progress
,include_workspace_size
andinclude_retention_rules
.
- Adds OpenAPI specification support for
-
Users:
- Adds support for specifying
kubernetes_job_timeout
for Kubernetes compute backend jobs. - Adds Kubernetes job memory limits validation before accepting workflows for execution.
- Adds support for HTML preview of workspace files in the web user interface.
- Adds an option to search for concrete file names in the workflow's workspace web user interface page.
- Changes the Cluster Health web interface page to display the cluster status information based on resource availability rather than only usage.
- Changes
info
command to include the list of supported compute backends. - Fixes workflow stuck in pending status due to early Yadage failures.
- Fixes formatting of error messages and sets appropriate exit status codes.
- Adds support for specifying
-
Administrators:
- Adds new configuration option to set default job timeout value for the Kubernetes compute backend jobs (
kubernetes_jobs_timeout_limit
). - Adds new configuration option to set maximum job timeout that users can assign to their jobs for the Kubernetes compute backend (
kubernetes_jobs_max_user_timeout_limit
). - Adds new configuration option
compute_backends
to specify the supported list of compute backends for validation purposes. - Adds new configuration option
reana_server.uwsgi.log_all
to toggle the logging of all the HTTP requests. - Adds new configuration options
reana_server.uwsgi.log_4xx
andreana_server.uwsgi.log_5xx
to only log HTTP error requests, i.e. HTTP requests with status code 4XX and 5XX. To make this configuration effectivereana_server.uwsgi.log_all
must befalse
. - Adds new configuration options
node_label_infrastructuremq
andnode_label_infrastructuredb
to have the possibility to run the Message Broker and the Database pods in specific nodes. - Changes uWSGI configuration to log all HTTP requests in REANA-Server by default.
- Changes
quota.disk_update
toquota.periodic_update_policy
to also update the CPU quota. Keepsquota.disk_update
for backward compatibility. - Changes the name of configuration option
quota.termination_update_policy
toquota.workflow_termination_update_policy
. Keepsquota.termination_update_policy
for backward compatibility.
- Adds new configuration option to set default job timeout value for the Kubernetes compute backend jobs (
-
Developers:
- Adds workflow name validation to the
create_workflow
endpoint, restricting special characters like dots. - Changes
/api/info
endpoint to return a list of supported compute backends. - Changes
/api/status
endpoint to calculate the cluster health status based on the availability instead of the usage. - Changes the way of determining Snakemake job statuses, polling the Job Controller API instead of checking local files.
- Adds workflow name validation to the
-
Users:
- Adds support for running and validating Snakemake workflows.
- Adds support for
outputs.directories
inreana.yaml
allowing to easily download output directories. - Adds new command
quota-show
to retrieve information about total CPU and Disk usage and quota limits. - Adds new command
info
that retrieves general information about the cluster, such as available workspace path settings. - Changes
validate
command to add the possibility to check the workflow against server capabilities such as desired workspace path via--server-capabilities
option. - Changes
list
command to add the possibility to filter by workflow status and search by workflow name via--filter
option. - Changes
list
command to add the possibility to filter and display all the runs of a given workflow via-w
option. - Changes
list
command to stop including workflow progress and workspace size by default. Please use new options--include-progress
and--include-workspace-size
to show this information. - Changes
list --sessions
command to display the status of interactive sessions. - Changes
logs
command to display also the start and finish times of individual jobs. - Changes
ls
command to add the possibility to filter by file name, size and last-modified values via--filter
option. - Changes
du
command to add the possibility filter by file name and size via--filter
option. - Changes
delete
command to prevent hard-deletion of workflows. - Changes Yadage workflow specification loading to be done in
reana-commons
. - Changes CWL workflow engine to
cwltool
version3.1.20210628163208
. - Removes support for Python 2.7. Please use Python 3.6 or higher from now on.
-
Administrators:
- Adds new configuration options
node_label_runtimebatch
,node_label_runtimejobs
,node_label_runtimesessions
allowing to set cluster node labels for splitting runtime workload into dedicated workflow batch nodes, workflow job nodes and interactive session nodes. - Adds new configuration option
workspaces.paths
allowing to set a dictionary of available workspace paths to pairs ofcluster_node_path:cluster_pod_mountpath
for mounting directories from cluster nodes. - Adds new configuration option
quota.enabled
to enable or disable CPU and Disk quota accounting for users. - Adds new configuration option
quota.termination_update_policy
to select the quota resources such as CPU and Disk for which the quota usage will be calculated immediately at the workflow termination time. - Adds new periodic cron job to update Disk quotas nightly. Useful if the
quota.termination_update_policy
does not include Disk quota resource. - Adds configuration environment variable
reana_server.environment.REANA_WORKFLOW_SCHEDULING_POLICY
allowing to set workflow scheduling policy (first-in first-out, user-balanced and workflow-complexity balanced). - Adds configuration environment variables
reana_server.environment.REANA_RATELIMIT_GUEST_USER
,reana_server.environment.REANA_RATELIMIT_AUTHENTICATED_USER
allowing to set REST API rate limit values. - Adds configuration environment variable
reana_server.environment.REANA_SCHEDULER_REQUEUE_SLEEP
to set a time to wait between processing queued workflows. - Adds configuration environment variable
reana_workflow_controller.environment.REANA_JOB_STATUS_CONSUMER_PREFETCH_COUNT
allowing to set a prefetch count for the job status consumer. - Adds support for Kubernetes 1.21 version clusters.
- Adds default
kubernetes_memory_limit
value (4 GiB) that will be used for all user jobs unless they specify otherwise. - Changes Helm template to use PostgreSQL 12.8 version.
- Changes Helm template for
reana-db
component to allow 300 maximum number of database connections by default. - Fixes email validation procedure during
create-admin-user
command to recognize more permissive email address formats.
- Adds new configuration options
-
Developers:
- Changes
git-*
commands to add the possibility of excluding certain components via the--exclude-components
option. - Changes
git-create-release-commit
command to bump all version files in a component. - Changes
git-log
command to show diff patch or to pass any wanted argument. - Changes
helm-upgrade-components
command to also upgrade the image tags inprefetch-images.sh
script.
- Changes
-
Users:
- Adds support for file listing wildcard matching patterns to
ls
command. - Adds support for directory download and wildcard matching patterns to
download
command. - Adds support for specifying
kubernetes_memory_limit
for Kubernetes compute backend jobs for CWL, Serial and Yadage workflows. - Changes
list
command to include deleted workflows by default. - Changes
validate
command to warn about incorrectly used workflow parameters for each step. - Changes
validate
command to display more granular workflow validation output. - Fixes workflow step job command formatting bug for CWL workflows on HTCondor compute backend.
- Fixes
validate
command output for verifying environment image UID values. - Fixes
upload_to_server()
Python API function to silently skip uploading in case of none-like inputs. - Fixes
validate
command for environment image validation to not test repetitively the same image found in different steps.
- Adds support for file listing wildcard matching patterns to
-
Administrators:
- Adds support for Kubernetes 1.21.
- Adds configuration environment variable to set default job memory limits for the Kubernetes compute backend (
REANA_KUBERNETES_JOBS_MEMORY_LIMIT
). - Adds configuration environment variable to set maximum custom memory limits that users can assign to their jobs for the Kubernetes compute backend (
REANA_KUBERNETES_JOBS_MAX_USER_MEMORY_LIMIT
). - Changes HTCondor compute backend to 8.9.11 and
myschedd
package and configuration to latest versions. - Fixes Kubernetes job log capture to include information about failures caused by external factors such as out-of-memory situations (
OOMKilled
).
-
Developers:
- Adds new functions to serialise/deserialise job commands between REANA components.
- Changes client dependencies to unpin six so that client may be installed in more contexts.
- Changes cluster dependencies to remove click and pins several dependencies.
- Changes
reana_ready()
function location to REANA-Server.
-
Users:
- Adds
reana-client validate
options to detect possible issues with workflow input parameters and environment images. - Fixes problem with failed jobs being reported as still running in case of network problems.
- Fixes job command encoding issues when dispatching jobs to HTCondor and Slurm backends.
- Adds
-
Administrators:
- Adds new configuration to toggle Kubernetes user jobs clean up.
: (
REANA_RUNTIME_KUBERNETES_KEEP_ALIVE_JOBS_WITH_STATUSES
incomponents.reana_workflow_controller.environment
) - Improves platform resilience.
- Adds new configuration to toggle Kubernetes user jobs clean up.
: (
-
Developers:
- Adds new command-line options to
reana-dev run-example
command allowing full parallel asynchronous execution of demo examples. - Adds default configuration for developer deployment mode to keep failed workflow and job pods for easier debugging.
- Changes job status consumer communications to improve overall platform resilience.
- Adds new command-line options to
-
Administrators:
- Adds support for deployments on Kubernetes 1.20 clusters.
- Adds deployment option to disable user email confirmation step after sign-up. (
REANA_USER_EMAIL_CONFIRMATION
incomponents.reana_server.environment
) - Adds deployment option to disable user sign-up feature completely. (
components.reana_ui.hide_signup
) - Adds deployment option to display CERN Privacy Notice for CERN deployments. (
components.reana_ui.cern_ropo
)
-
Developers:
- Adds support for Python 3.9.
- Fixes minor code warnings.
- Changes CI system to include Python flake8 and Dockerfile hadolint checkers.
-
Users:
- Adds support for specifying
htcondor_max_runtime
andhtcondor_accounting_group
for HTCondor compute backend jobs. - Fixes restarting of Yadage and CWL workflows.
- Fixes REANA <-> GitLab synchronisation for projects having additional external webhooks.
- Changes
ping
command output to include REANA client and server version information.
- Adds support for specifying
-
Developers:
- Fixes conflicting
kombu
installation requirements by requiring Celery version 4. - Changes
/api/you
endpoint to include REANA server version information. - Changes continuous integration platform from Travis CI to GitHub Actions.
- Fixes conflicting
-
Users:
- Adds new
restart
command to restart previously run or failed workflows. - Adds option to
logs
command to filter job logs according to compute backend, docker image, job status and step name. - Adds option to specify operational options in the
reana.yaml
of the workflow. - Adds option to specify unpacked Docker images as workflow step requirement.
- Adds option to specify Kubernetes UID for jobs.
- Adds support for VOMS proxy as a new authentication method.
- Adds support for pulling private Docker images.
- Adds pagination on the workflow list and workflow detailed web interface pages.
- Adds user profile page to the web interface.
- Adds page refresh button to workflow detailed page.
- Adds local user web forms for sign-in and sign-up functionalities for local deployments.
- Fixes user experience by preventing dots as part of the workflow name to avoid confusion with restart runs.
- Fixes workflow specification display to show runtime parameters.
- Fixes file preview functionality experience to allow/disallow certain file formats.
- Changes Yadage workflow engine to version 0.20.1.
- Changes CERN HTCondor compute backend to use the new
myschedd
connection library. - Changes CERN Slurm compute backend to improve job status detection.
- Changes documentation to move large parts to docs.reana.io.
- Changes
du
command output format. - Changes
logs
command to enhance formatting using marks and colours. - Changes
ping
command to perform user access token validation. - Changes
diff
command to improve output formatting. - Changes defaults to accept both
reana.yaml
andreana.yml
filenames. - Changes from Bravado to requests to improve download performance.
- Changes file loading to optimise CLI performance.
- Adds new
-
Administrators:
- Adds Helm chart and switches to Helm-based deployment technique instead of using now-deprecated
reana-cluster
. - Adds email notification service to inform administrators about system health.
- Adds announcement configuration option to display any desired text on the web UI.
- Adds pinning of all Python dependencies allowing to easily rebuild component images at later times.
- Adds support for local user management and web forms for sign-in and sign-up functionalities.
- Adds support for database upgrades using Alembic.
- Changes installation procedures to move database initialisation and admin creation after Helm installation.
- Changes service exposure to stop exposing unused Invenio-Accounts views.
- Changes runtime job instantiation into the configured runtime namespace.
- Changes CVMFS to be read-only mount.
- Adds Helm chart and switches to Helm-based deployment technique instead of using now-deprecated
-
Developers:
- Adds several new
reana-dev
commands to help with merging, releasing, unit testing. - Changes base image to use Python 3.8 for all REANA cluster components.
- Changes pre-requisites to node version 12 and latest npm dependencies.
- Changes back-end code formatting to respect
black
coding style. - Changes front-end code formatting to respect updated
prettier
version coding style. - Changes test strategy to start PostgreSQL DB container to run tests locally.
- Changes auto-generated component documentation to single-page layout.
- Adds several new
-
Administrators:
- Fixes installation troubles for REANA 0.6.x release series by pinning several dependencies.
- Upgrades REANA-Commons package to latest Kubernetes Python client version.
- Amends documentation for
minikube start
to include VirtualBox hypervisor explicitly.
-
Users:
- Adds support for HTCondor compute backend for all workflow engines (CWL, Serial, Yadage).
- Adds support for Slurm compute backend for all workflow engines (CWL, Serial, Yadage).
- Allows to run hybrid analysis pipelines where different parts of the workflow can run on different compute backends (HTCondor, Kubernetes, Slurm).
- Adds support for Kerberos authentication mechanism for user workflows.
- Introduces user secrets management commands
secrets-add
,secrets-list
andsecrets-delete
. - Fixes
upload
command behaviour for uploading very large files. - Upgrades CWL workflow engine to 1.0.20191022103248.
- Upgrades Yadage workflow engine to 0.20.0 with Packtivity 0.14.21.
- Adds support for Python 3.8.
- See additional changes in reana-client 0.6.0 release notes.
-
Administrators:
- Upgrades to Kubernetes 1.16 and moves Traefik installation to Helm 3.0.0.
- Creates a new Kubernetes service account for REANA with appropriate permissions.
- Makes database connection details configurable so that REANA can connect to databases external to the cluster.
- Autogenerates deployment secrets if not provided by administrator at cluster creation time.
- Adds an interactive mode on cluster initialisation to allow providing deployment secrets.
- Adds CERN specific Kerberos configuration files and CERN EOS storage support.
- See additional changes in reana-cluster 0.6.0 release notes.
-
Developers:
- Modifies the batch workflow runtime pod creation including an instance of job controller running alongside workflow engine using the sidecar pattern.
- Adds generic job manager class and provides example classes for CERN HTCondor and CERN Slurm clusters.
- Provides user secrets to the job container runtime tasks.
- Adds sidecar container to the Kubernetes job pod if Kerberos authentication is required.
- Refactors job monitoring using the singleton pattern.
- Enriches
make
behaviour for developer-oriented installations with live code reload changes and debugging. - Enriches
git-status
component status reporting for developers. - See additional changes in individual REANA 0.6.0 platform components.
-
Users:
- Allows to explore workflow results by running interactive Jupyter notebook sessions on the workspace files.
- Allows to declare computing resources needed for workflow runs, such as access to CVMFS repositories.
- Improves
reana-client
command-line client with new options to stop workflows, diff workflows, move and remove files. - Upgrades CWL engine to 1.0.20181118133959.
- See additional changes in reana-client 0.5.0 release notes.
-
Administrators:
- Upgrades to Kubernetes 1.14, Helm 2.13 and Minikube 1.0.
- Separates cluster infrastructure pods from runtime workflow engine pods that will be created by workflow controller.
- Introduces configurable CVMFS and CephFS shared volume mounts.
- Adds support for optional HTTPS protocol termination.
- Introduces incoming workflow queue for additional safety in case of user storms.
- Makes infrastructure pods container image slimmer to reduce the memory footprint.
- See additional changes in reana-cluster 0.5.0 release notes.
-
Developers:
- Enhances development process by using git-submodule-like behaviour for shared components.
- Introduces simple Makefile for (fast) local testing and (slow) nightly building purposes.
- Centralises logging level and common Celery tasks.
- Adds helpers for test suite fixtures and improves code coverage.
- See additional changes in individual REANA 0.5.0 platform components.
- Uses common OpenAPI client in communications between workflow engines and job controller.
- Improves AMQP re-connection handling.
- Enhances test suite and increases code coverage.
- Changes license to MIT.
- Introduces new Serial workflow engine for simple sequential workflow needs.
- Enhances progress reporting for CWL, Serial and Yadage workflow engines.
- Simplifies
reana-client
command set and usage scenarios. - Introduces multi-user capabilities with mandatory access tokens.
- Adds support for multi-node clusters using shared CephFS volumes.
- Adds support for Kubernetes 1.11, Minikube 0.28.2.
- Upgrades CWL workflow engine to use latest
cwltool
version. - Fixes several bugs such as binary file download with Python 3.
- Adds support for Common Workflow Language workflows.
- Adds support for persistent user-selected workflow names.
- Enables file and directory input uploading using absolute paths.
- Enriches
reana-client
andreana-cluster
command set. - Reduces verbosity level for commands and improves error messages.
- Initial public release.