diff --git a/.gitignore b/.gitignore index 89502188..88011228 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ _build _bikeshed .tox .vscode +.*plist diff --git a/0.1/index.bs b/0.1/index.bs index 7cb79539..e044a74f 100644 --- a/0.1/index.bs +++ b/0.1/index.bs @@ -364,7 +364,7 @@ custom attributes of the plate group under the `plate` key. For example the following JSON object defines a plate with two acquisition and -6 wells (2 rows and 3 columns), containing up 2 fields of view per acquistion. +6 wells (2 rows and 3 columns), containing up 2 fields of view per acquisition. ```json "plate": { diff --git a/0.2/index.bs b/0.2/index.bs index e8e5a7bd..64c07c78 100644 --- a/0.2/index.bs +++ b/0.2/index.bs @@ -419,7 +419,7 @@ custom attributes of the plate group under the `plate` key. For example the following JSON object defines a plate with two acquisition and -6 wells (2 rows and 3 columns), containing up 2 fields of view per acquistion. +6 wells (2 rows and 3 columns), containing up 2 fields of view per acquisition. ```json "plate": { diff --git a/0.3/index.bs b/0.3/index.bs index ed424f62..0f64fbe0 100644 --- a/0.3/index.bs +++ b/0.3/index.bs @@ -432,7 +432,7 @@ custom attributes of the plate group under the `plate` key. For example the following JSON object defines a plate with two acquisition and -6 wells (2 rows and 3 columns), containing up 2 fields of view per acquistion. +6 wells (2 rows and 3 columns), containing up 2 fields of view per acquisition. ```json "plate": { diff --git a/_static/main.js b/_static/main.js new file mode 100644 index 00000000..fc7a88ee --- /dev/null +++ b/_static/main.js @@ -0,0 +1,3 @@ +$(document).ready( function () { + $('table.datatable').DataTable(); +} ); diff --git a/conf.py b/conf.py index 3b941302..f18fb755 100644 --- a/conf.py +++ b/conf.py @@ -15,6 +15,8 @@ extensions = ["myst_parser"] source_suffix = [".rst", ".md"] +myst_heading_anchors = 5 +myst_enable_extensions = ["deflist"] templates_path = ['_templates'] exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', '.git', '.pytest_cache', '**/.pytest_cache', '**/.tox', 'README.md', 'LICENSE.md', 'CONTRIBUTING.md'] diff --git a/data/index.md b/data/index.md index f4cdcec0..be0f179e 100644 --- a/data/index.md +++ b/data/index.md @@ -7,6 +7,7 @@ Data Resources | [Cell Painting Gallery](https://github.com/broadinstitute/cellpainting-gallery) | AWS Open Data Program | 136 | 20 TB | | [CZB-Zebrahub](https://zebrahub.ds.czbiohub.org/imaging) | czbiohub | 5 | 1.2 TB | | [DANDI](https://dandiarchive.org/dandiset/000108) ([identifiers.org][dandi2],[github][dandi3]) | AWS Open Data Program | 3914 | 355 TB | +| [JAX](https://images.jax.org/webclient/userdata/?experimenter=-1) | The Jackson Laboratory | 17000 | 192 TB | | [Glencoe](https://glencoesoftware.com/ngff) | Glencoe Software, Inc. | 8 | 165 GB | | [IDR Samples](https://idr.github.io/ome-ngff-samples/) | EBI | 88 | 3 TB | | [MoBIE](https://mobie.github.io/specs/ngff.html) | EMBL-HD | 21 | 2 TB | diff --git a/index.rst b/index.rst index 1b15fb9c..ecbb8bab 100644 --- a/index.rst +++ b/index.rst @@ -24,6 +24,3 @@ Sample NGFF datasets provided by the community can be found under :doc:`data/ind specifications/index rfc/index tools/index - -.. raw:: html - diff --git a/requirements.txt b/requirements.txt index e43d0c22..9cc69f43 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,3 @@ -bikeshed +bikeshed<4.2 myst-parser sphinx-book-theme diff --git a/rfc/0/index.md b/rfc/0/index.md index 5c2549b8..cbe48ffb 100644 --- a/rfc/0/index.md +++ b/rfc/0/index.md @@ -1,4 +1,166 @@ -RFC-0 -===== +# RFC-0: Consensus model -Example +Original NGFF consensus process + +## Status + +This is a historical RFC, drafted after the fact, and has +been outdated by [RFC-1][1]. + +```{list-table} Record +:widths: 20, 20, 20, 15, 10 +:header-rows: 1 +:stub-columns: 1 + +* - Name + - GitHub Handle + - Institution + - Date + - Status +* - Josh Moore + - [joshmoore](https://github.com/joshmoore) + - [German BioImaging, e.V.](https://ror.org/05tpnw772) + - 2022-07-11 ([issue-132](https://github.com/ome/ngff/issues/132)) / 2024-08-30 (RFC-0) + - Author +``` + +## Overview + +Versions of the NGFF specification up to and including v0.4 followed a +consensus model of decision making. This process is being recorded as RFC-0 +to simplify referencing the former process in [RFC-1][2] and elsewhere. + +## Background + +NGFF development began as a relatively focused group of individuals writing +specifications. At the time, this group was called "editors". This document +describes the process that was used for reviewing and finding a consensus on +their work before [RFC-1][3] was adopted. This period started with the +first commit 2020-11-17 and covers versions 0.1 through 0.4, while +version 0.5 was largely developed concurrently with [RFC-1][4]. + +## Proposal + +A general outline of how versions were decided: + +* **Proposals** generally gathered initial support during **community calls** which often led to the drafting of an issue. In the case of v0.1, + the first issue was on the zarr-specs repository, + [https://github.com/zarr-developers/zarr-specs/issues/50][5]. +* To the extent possible, **implementations** were started as soon as there was + a general consensus since it is much easier to show what is intended when + there is draft data which requires a writer implementation. This data was + often uploaded to a temporary S3 location. +* Once the **PR was opened**, there was a general call for **feedback** + typically via the [https://image.sc][6] **@ngff group** with various iterations. + Sometimes a second **community call** occurred at this point. +* Once no further changes were requested, a final call with a deadline for + release was made either via the image.sc group or a community call or both. +* The tag was pushed and then an attempt to make sure all implementations were + aligned followed. + +An exception to the above were a small number of "transitional" specifications. +This includes the `omero` and `bioformats2raw.layout` sections. They differ +from other proposals in that: + + - They captured a format that was already in use in the wild. + - They affected one or more _existing_ versions. + - Support in all clients seemed to be less essential, though there was no precise definition of “essential” . + +## Requirements + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in IETF RFC 2119. + +## Stakeholders + +* **editors**: those authored specifications. Note: this terminology + differs from that in [RFC-1][7] and the website and specifications have been updated to reflect this change. +* **Implementers**: those who maintain a library or client which implements specs (see below) +* **developer community**: those who build tools that are expected to support NGFFs (e.g., napari, BDV, Viv) +* **[the ngff group][8]**: those who have actively signed up for calls, feedback, etc. +* and, of course, the wider community of users. + +## Implementation + +Typically a specification proposal included a single PR which updated the `latest` version +of the specification in the `index.bs` file and additionally included examples and schema +files representing the changes. + +Once data could be generated by one of the implementations, there was then a +general attempt to update all implementing libraries (ome-zarr-py, n5-zarr, +[validator][9], etc.) and implementing +clients (MoBIE, ViZarr, napari-ome-zarr, neuroglancer, ITK, etc.). Minimally, +one update in each of the three primary languages, Java, Python and Javascript, +was undertaken. + + +### Phases + +The phases below were sometimes concurrent and/or in a different order. + +* **Proposal**: an informal statement that someone would like to lead an effort + to change the specification. +* **Issue opened** (optional): a slightly more formal statement of the purposes of a + Proposal that tracked all of the associated work. +* **Community call**: an open call, usually on Zoom, announced under + [https://forum.image.sc/tag/ome-ngff][10] where a proposal might be discussed. +* **Hackathon**: an open, often in-person, meeting to work on a Proposal. +* **Drafting**: period during which the specification is being actively + modified. +* **PR Opened**: a change to the specification including the normative text, + examples, schemas and schema tests. +* **Discussions**: once a formal change was opened, period of time during + which comments and feedback were collected and changes made. +* **PR Merged**: no further changes intended and the specification is + considered acceptable to be part of "latest". +* **Acceptance**: the changes to "latest" are considered acceptable for a + release and are ported to the appropriate version directory. +* **Implementation**: support added to each of the libraries and clients + listed above. A differentiation MAY be made between core (MUST) and + additional (SHOULD) components. +* **Release**: an NGFF specification version was formally released by adding the + appropriate tag to the NGFF repository. + +## Drawbacks, risks, alternatives, and unknowns + +Beyond a certain scale, the consensus model does not help maintainers of the +specification to move discussions forward. There is no clear arbiter of +decisions, and the timeline of discussions are not specified. A decision may +have been made, but without a clear record pull requests open for +implementation are effectively still in play. Additionally, authors were often +left with many separate questions on GitHub pull requests that needed answering +with no clear mechanism for managing those. + +## Future possibilities + +Please see [RFC-1][11]. + +## Examples + +The following PRs updated the specification under the RFC-0 model: +* [3][12] labels (v0.1.3) +* [5][13] high-content-screening (v0.1.4) +* [40][14] nested storage (0.2.0) +* [46][15] and [57][16] axes (0.3.0) +* [64][17] table (closed) +* [112][18] transitional bioformats2raw (0.4.1) + +[1]: ../1/index +[2]: ../1/index +[3]: ../1/index +[4]: ../1/index +[5]: https://github.com/zarr-developers/zarr-specs/issues/50 +[6]: https://image.sc +[7]: ../1/index +[8]: https://forum.image.sc/g/ngff +[9]: https://github.com/ome/ome-ngff-validator/ +[10]: https://forum.image.sc/tag/ome-ngff +[11]: ../1/index +[12]: https://github.com/ome/ngff/pull/3 +[13]: https://github.com/ome/ngff/pull/5 +[14]: https://github.com/ome/ngff/pull/40 +[15]: https://github.com/ome/ngff/pull/46 +[16]: https://github.com/ome/ngff/pull/57 +[17]: https://github.com/ome/ngff/pull/64 +[18]: https://github.com/ome/ngff/pull/112 diff --git a/rfc/1/comment_1.md b/rfc/1/comments/1/index.md similarity index 98% rename from rfc/1/comment_1.md rename to rfc/1/comments/1/index.md index 1d5d8cbc..336343cf 100644 --- a/rfc/1/comment_1.md +++ b/rfc/1/comments/1/index.md @@ -1,4 +1,4 @@ -# Comment to [RFC 1](../1) from @LucaMarconato and @melonora.* +# RFC-1: Comment 1 | Name | GitHub Handle | Institution | |------------------------|---------------|----------------------| diff --git a/rfc/1/comment_2.md b/rfc/1/comments/2/index.md similarity index 98% rename from rfc/1/comment_2.md rename to rfc/1/comments/2/index.md index 647a7dee..37944c92 100644 --- a/rfc/1/comment_2.md +++ b/rfc/1/comments/2/index.md @@ -1,4 +1,4 @@ -# Comment to [RFC 1](../1) from @thewtex +# RFC-1: Comment 2 | Name | GitHub Handle | Institution | |------------------------|---------------|----------------------| diff --git a/rfc/1/comments/index.md b/rfc/1/comments/index.md new file mode 100644 index 00000000..428a7147 --- /dev/null +++ b/rfc/1/comments/index.md @@ -0,0 +1,9 @@ +# Comments + +Additional comments of RFC-1: + +```{toctree} +:maxdepth: 1 +:glob: +*/index +``` diff --git a/rfc/1/diagram.png b/rfc/1/diagram.png index bd99cca5..70fa9228 100644 Binary files a/rfc/1/diagram.png and b/rfc/1/diagram.png differ diff --git a/rfc/1/drawing.png b/rfc/1/drawing.png new file mode 100644 index 00000000..1db757a9 Binary files /dev/null and b/rfc/1/drawing.png differ diff --git a/rfc/1/index.md b/rfc/1/index.md index 2e489772..83d60402 100644 --- a/rfc/1/index.md +++ b/rfc/1/index.md @@ -1,84 +1,201 @@ -RFC-1 -===== - -Definition of the NGFF RFC process +RFC-1: RFC Process +================== + +Definition of the NGFF “Request for Comment” (RFC) process + +```{toctree} +:hidden: +:maxdepth: 1 +templates/index +reviews/index +comments/index +responses/index +versions/index +``` ## Status -This RFC is currently being responded to (R4). - -| Name | GitHub Handle | Institution | Date | Status | -| ---------------------------------------- | ------------------------- | ---------------------- | ------------ | --------------------------------------------------- | -| Josh Moore | joshmoore | German BioImaging | 2023-12-23 | Author ([PR](https://github.com/ome/ngff/pull/222)) | -| Virginia Scarlett, _et al._ | | Janelia | 2024-02-26 | [Review](./review_2.md) | -| Kevin Yamauchi, Virginie Uhlmann | | ETH, BiovisionCenter | 2024-03-05 | [Review](./review_1.md) | -| Matthew Hartley | | EMBL-EBI | 2024-03-05 | [Review](./review_3.md) | -| Wouter-Michiel Vierdag, Luca Marconato | melonora, LucaMarconato | EMBL | 2024-01-13 | [Comment](./comment_1.md) | -| Matt McCormick | thewtex | ITK | 2024-01-09 | [Comment](./comment_2.md) | +This RFC has been adopted (S4). + +```{list-table} Record +:widths: 8, 20, 20, 20, 15, 10 +:header-rows: 1 +:stub-columns: 1 + +* - Role + - Name + - GitHub Handle + - Institution + - Date + - Status +* - Author + - Josh Moore + - [joshmoore](https://github.com/joshmoore) + - [German BioImaging, e.V.](https://ror.org/05tpnw772) + - 2023-12-23 + - Author ([PR](https://github.com/ome/ngff/pull/222)) +* - Reviewer + - Davis Bennett, John Bogovic, Michael Innerberger, Mark Kittisopikul, Virginia Scarlett, Yurii Zubov + - d-v-b, bogovicj, minnerbe, mkitti, virginiascarlett, yuriyzubov + - Janelia + - 2024-02-26 + - [Review](./reviews/2/index) +* - Reviewer + - Kevin Yamauchi, Virginie Uhlmann + - kevinyamauchi, vuhlmann + - ETH, BiovisionCenter + - 2024-03-05 + - [Review](./reviews/1/index) +* - Reviewer + - Matthew Hartley + - mrmh2 + - EMBL-EBI + - 2024-03-05 + - [Review](./reviews/3/index) +* - Commenter + - Wouter-Michiel Vierdag, Luca Marconato + - melonora, LucaMarconato + - EMBL + - 2024-01-13 + - [Comment](./comments/1/index) +* - Commenter + - Matt McCormick + - thewtex + - ITK + - 2024-01-09 + - [Comment](./comments/2/index) +* - Author + - Josh Moore + - joshmoore + - German BioImaging + - 2024-08-29 + - [Response](./responses/1/index) +* - Endorser + - Jean-Marie Burel + - jburel + - University of Dundee + - 2024-09-09 + - [Endorse](https://github.com/ome/ngff/pull/258) +* - Endorser + - Will Moore + - will-moore + - University of Dundee + - 2024-09-09 + - [Endorse](https://github.com/ome/ngff/pull/258) +* - Endorser + - Juan-Nuñez Iglesias + - jni + - Monash University + - 2024-09-09 + - [Endorse](https://github.com/ome/ngff/pull/258) +* - Endorser + - Norman Rzepka + - normanrz + - scalable minds GmbH + - 2024-09-09 + - [Endorse](https://github.com/ome/ngff/pull/258) +* - Endorser + - Jan Eglinger + - imagejan + - FMI Basel + - 2024-09-09 + - [Endorse](https://github.com/ome/ngff/pull/258) +* - Endorser + - Joel Lüthi + - jluethi + - BioVisionCenter, University of Zurich + - 2024-09-10 + - [Endorse](https://github.com/ome/ngff/pull/258) +* - Reviewer + - Kevin Yamauchi, Joel Lüthi, Virginie Uhlmann + - kevinyamauchi, jluethi, vuhlmann + - ETH, BiovisionCenter + - 2024-10-03 + - [Accept](./reviews/1b/index) +* - Reviewer + - Matthew Hartley + - mrmh2 + - EMBL-EBI + - 2024-10-08 + - Accept (email) +* - Reviewer + - John Bogovic, Michael Innerberger, Virginia Scarlett + - bogovicj, minnerbe, virginiascarlett + - Janelia + - 2024-10-11 + - [Accept](./reviews/2b/index) +``` ## Overview This RFC defines the high-level decision making process for changes within the NGFF community. These changes are defined in "Request for Comments" (RFCs) and therefore this RFC is self-referential: it is following the process that it -itself defines. The overall goal of the process is to make clear how decision -making works with a focus on both speed of development and clarity. It should -be clear after reading the RFC which stakeholder (author, reviewer, editor, -etc.) is responsible for moving decisions along and how much time the -community can expect that decision to take. Not all decisions in the NGFF -community need to follow the RFC process. Clarifications, corrections, and -numerous other changes will proceed following the current GitHub workflow. -However, when decisions reach a certain scale, including significant -specification changes but also changes to the community process itself, RFCs -will provide a mechanism for managing the process. +itself defines. ## Background Growing interest in the NGFF format has also led to increased participation in the specification process. As such, reaching a consensus for all decisions has -become more difficult. The current approach defined post facto in RFC-0 +become more difficult. The current approach defined _post facto_ in RFC-0 follows a full consensus model. Through community meetings and pull requests, it was expected that all parties agree before a specification change was -considered. This made it especially difficult both for **Authors** as well as -**Reviewers** to know when a suggested change was adopted since PR comments -could re-open a discussion. It also left a significant burden on **Editors** to +considered. This made it especially difficult both for those proposing as well as those reviewing changes to know when a suggested change was adopted since PR comments +could re-open a discussion. It also left a significant burden on maintainers to draw a line where further discussion was not possible. Such a situation leads to slower specification evolution and potential deadlocks. +Additionally, extended GitHub discussions had become difficult to follow. Without some curation of the content, it was unclear to those not deeply involved from the beginning to see what decisions had been made, by whom, when and why. There were also limited options for managing the often complex conversions and differing opinions that arise in technical conversations. + Led by the need to take on larger challenges and more interested parties, there has been significant interest within the community to update the process used -in order to bypass these issues. This RFC adapts the well-known [RFC process][1], +in order to bypass these issues. This RFC adapts the well-known [RFC process](https://www.rfc-editor.org/), which originated in the Internet Engineering Task Force (IETF), for use in the -NGFF community as has been done in a number of other communities ([Rust][2], -[Hashicorp][3], [Tensorflow][4], etc.) More information can be found under: - -- -- +NGFF community as has been done in a number of other communities ([Rust](https://github.com/rust-lang/rfcs/blob/master/0000-template.md), +[Hashicorp](https://works.hashicorp.com/articles/rfc-template), [Tensorflow](https://github.com/tensorflow/community/blob/master/rfcs/yyyymmdd-rfc-template.md), etc.) More information can be found under: -Ultimately, each RFC is a record of a decision made, either for or against a -proposal, that will be available from the main NGFF webpage. This captures the -currently distributed GitHub conversations into a single, consistent location -that can be reviewed and referenced more widely by the community. +- [https://en.wikipedia.org/wiki/Internet\_Standard#Standardization\_process](https://en.wikipedia.org/wiki/Internet_Standard#Standardization_process) +- [https://en.wikipedia.org/wiki/Request\_for\_Comments](https://en.wikipedia.org/wiki/Request_for_Comments) ## Proposal -The RFC process proposed functions by (1) encouraging submissions from the -community and not blocking their encapsulation as an RFC (2) asking for -descriptive and complete comments from reviewers on these submissions (3) -having clear definitions of what support **must** be received for decisions to -move forward. Goals of this process include (a) maintaining a public record of -the decision making (b) querying acceptance of a proposal throughout the -process and making that readily visible (c) ultimately driving the +Requests for Comment (RFCs) are intended to structure high-level discussions on changes within the NGFF community and record outcomes including key opinions, actions, and decisions. The overall goal of the process is timely and transparent decision-making for a stable and trusted community specification. It should be clear after reading the RFC which stakeholder (**Author**, **Reviewer**, **Editor**, etc.) is responsible for each step of the process, what options are available to the decision makers, and how much time the community can expect that decision to take. + +![Simplified drawing of the RFC process](./drawing.png) + +**Figure 1. Simplified drawing of the RFC process** An RFC draft (1) is +proposed by **Authors** who would like to see some change in the NGFF +community. There is a period of gathering _[endorsements](#def-endorsement)_ +(2) which will be listed in the RFC itself. This gives future readers and +especially those tasked with _implementing_ any changes more confidence in the +proposal itself. Once any necessary clarifications have been addressed, +**Editors** will assign the RFC a number and send it to **Reviewers** for +detailed responses (3). These are captured in the RFC directory. **Authors** +will then update the RFC with any needed changes and provide a _response_ to +the **Reviewers** (4). Once this iterative process of review is complete, the +RFC will either be approved or not. Once approved, the specification is updated +with the RFC changes (5). After a minimum number of implementations have been +achieved, the RFC is considered _adopted_ (6). + +The RFC process functions by encouraging submissions from the +community that are recorded for posterity *even if not adopted*. +Descriptive and complete comments from both **Authors** and **Reviewers** are critical +to have a clear understanding of what decisions have been made. +Goals of this process include maintaining a public record of +the decision making, querying endorsement of a proposal throughout the +process and making that readily visible, and ultimately driving the implementation of specifications in order to get working solutions into the hands of the bioimaging community. The process should **NOT** prevent future discussions on any adopted RFCs but instead will encourage continued improvement and evolution through discussions of _further_ RFCs. +Additionally, not all decisions in the NGFF community need to follow the RFC process. Clarifications, corrections, and numerous other changes will proceed following the current GitHub workflow. However, when decisions reach a certain scale, including significant specification changes but also changes to the community process itself, RFCs will provide a mechanism for managing the process. + ## Requirements The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be -interpreted as described in IETF RFC 2119. +interpreted as described in IETF [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). ## Stakeholders @@ -87,50 +204,48 @@ overview of their responsibilities and involvement in the process. For more details, see the "Implementation" section below. **Authors** propose an idea for the RFC process and socialize the idea, e.g., -through an issue or community call, gaining **Endorsers** They then submit a -pull request to the repository with a document that +through an issue or community call, gaining **Endorsers.** They then submit a +pull request to the [https://github.com/ome/ngff](https://github.com/ome/ngff) repository with a document that they would like to have published as an RFC. This pull request MUST contain a document under the `rfc/` subdirectory and it SHOULD follow the template provided. As described under the "DRAFT" section below, this document can be discussed for clarity following the standard PR process. However, once the -draft has reached a certain stage that it is ready for comments, **Editors** +draft has reached a stage where it is ready for review, **Editors** will merge it as a record of the fact that the suggestion has been made, and it will then become available on https://ngff.openmicroscopy.org. -**Endorsers** are non-**Author** supporters of an RFC, listed in a table. -**Reviewers** who have given an "Accept" recommendation are also added to the -table. Rather than a process terminated with a single vote, the RFC process -consists of iterative rounds of gathering **Endorsers**, or sponsors, who -approve the work, building confidence that the community is moving in the -right direction. More information on this strategy can be found under: - . +**Endorsers** are non-**Author** supporters of an RFC, listed in a table within the RFC. +**Reviewers** who have given an "Accept" recommendation and **Implementers** are also considered **Endorsers**. -**Editors** identify whether a PR should or should not follow the RFC process, -and choosing when a draft is ready to become an RFC. They also choose +**Editors** are responsible for facilitating all parts of the RFC process. +They identify whether a PR should or should not follow the RFC process, +and choose when a draft is ready to become an RFC. They also choose appropriate **Reviewers** for an RFC and manage the communication between **Authors** and **Reviewers**. -Note: The use of "Editors" in this document is intended as a placeholder. A -future RFC will define the selection and removal of editors. Until that time, -the sole editor is Josh Moore which has been the _de facto_ case since the -inception of NGFF. +**Implementers** are responsible for an implementation of the NGFF specification in one or more programming languages. It is critical that specification RFCs have been evaluated by **Implementers** which is often best done in the implementation rather than a review. Therefore, statements that an RFC is “planned”, “begun”, or “complete” for an implementation will be given similar weight to an endorsement or positive review. **Reviewers** represent experts in the community whose opinion on an RFC is necessary in order for the community to feel comfortable that a solution is advantageous for and implementable by the community. Some number of reviewers will be asked to provide a review of the RFC along with a recommendation on how the **Editors** should proceed with the RFC. Depending on the complexity of the -RFC, this process may be iterative and require multiple reviews. +RFC, this process may be iterative and require multiple reviews. Rather than a process terminated with a single vote, the RFC process +consists of iterative rounds of gathering **Endorsers**, or sponsors, who +approve the work, building confidence that the community is moving in the +right direction. **Reviewers** should strive to provide feedback which informs **Authors** of the changes that are needed to reach acceptance. More information on this strategy can be found under: +[https://engineering.squarespace.com/blog/2019/the-power-of-yes-if](https://engineering.squarespace.com/blog/2019/the-power-of-yes-if) . -**Commenters** are other members of the community who are not required to make -a determination on an RFC but are still invited to add their voice to the -decision-making process. +**Commenters** are other members of the community who, though not contacted as **Reviewers**, have provided feedback that they would like added to the official record of the RFC. ## Implementation -This description of the RFC process will refer frequently to the [visual diagram][dia] -Readers may want to familiar themselves with it at this point. -Identifiers such as "D1", "R2", "S3", refer to steps in that diagram. +The RFC process can be represented as a state diagram with the various stakeholders responsible for forward motion. + +![State diagram of the RFC process](./diagram.png) + +**Figure 2. State diagram of the RFC process.** The RFC text will refer frequently to this diagram. Readers may want to familiarize themselves with it at this point. +Identifiers such as "D1", "R2", "S3", refer to individual steps. ### Legend @@ -143,42 +258,45 @@ with the following symbols: ### Phases -The overall process is broken down into three phases: DRAFT phase before a +The overall process is broken down into three phases: the DRAFT phase before a proposal becomes an RFC, the RFC phase during which reviews occur, and the SPEC -phase after the RFC has been adopted. +phase after the RFC has been accepted. #### DRAFT +**Purpose**: During the DRAFT phase, **Authors** discuss and develop an idea to maximize its potential as an RFC. + The DRAFT phase begins when **Authors** propose (D1) a new idea and subsequently gather support (or "socialize") the idea (D2) before opening a PR -(D3). This phase is meant to promote ideation without premature criticism, though +(D3) which SHOULD follow the [current template](templates/rfc_template.md). +This phase is meant to promote ideation without premature criticism, though clarifications can be requested on the GitHub PR (D4) which may require an update of the PR (D3). Note: the RFC process does not formally make use of GitHub's "draft" state, though **Editors** MAY choose to ignore such PRs. -One common clarification is whether or not a given PR should be an RFC. The -RFC process is intended primarily for significant and/or breaking changes to -the specification or the community process itself. Other changes, bug fixes, +The RFC process is intended primarily for significant and/or breaking changes to +the specification or the community process itself, as described under +["RFC Prioritization"](#rfc-prioritization). Other changes, bug fixes, etc. are welcome as PRs without a full RFC. If **Authors** are unsure whether or not a change requires an RFC, they can open an issue or contact the **Editors** -directly. All RFC-targeted PRs SHOULD follow the [current template][tmpl]. +directly. Comments on how a specification is implemented or what could be proposed instead from potential **Reviewers** and **Commenters** should come in the RFC stage, and the **Editor** MAY postpone such discussions when appropriate. -> 🕑 The iterative clarification phase should be limited to at most a month. +> 🕑 The iterative clarification phase can be expected to take on the order of a month. At the **Editors** discretion (D5), the PR can be merged at which point it becomes an RFC or closed if there is no interest in progressing with the -proposal. In the latter case, **Authors** can use any feedback to open a new PR +proposal (D6). In the latter case, **Authors** can use any feedback to open a new PR (D3). **Authors** who are unsure if they will be able to shepherd an RFC throughout the entire process are still invited to open PRs so that they might be adopted by other members of the community. These steps are not significantly different from the previous consensus model -described in RFC-0. However, it is intended that the discussions during this -period are intended to improve the RFC and is not intended to evaluate its +described in [RFC-0](../0/index). However, discussions during this +period are intended to improve the RFC and not to evaluate its overall value. As described in the next section "RFC", the deeper and more critical discussions should happen as complete and well-considered reviews and responses that will help future readers understand the decision-making @@ -188,20 +306,23 @@ silence anyone but to manage the overall flow of discussion. #### RFC -Once a PR has been merged to become an RFC, **Editors** are responsible for +**Purpose**: During the RFC phase, an iterative process of community endorsement and/or feedback helps to achieve a consensus decision. + +Once a PR has been merged and has been assigned an RFC number, **Editors** are responsible for +adjusting the "Status" in the RFC document to become "RFC phase (R1)", and identifying and assigning **Reviewers** (R1). **Reviewers** will be given a period of time for preparing their reviews. If they do not foresee being able to respond in this time, they SHOULD contact the **Editors** as soon as possible. -> 🕑 Reviewers comments should be returned in less than one month. +> 🕑 Reviewers comments SHOULD be returned in less than one month. **Reviewers** then submit their comments (R2) along with their recommendations to the **Editors**, either via a public PR adding the review in markdown to the RFC's subdirectory or by emailing the **Editors** directly. (This latter course -should only be limitedly used when necessary.) +should only be used when necessary.) -Possible recommendations from **Reviewers** in ascending order of support are: +(rfc-recommendations)= Possible recommendations from **Reviewers** in ascending order of support are: * “Reject” suggests that a **Reviewer** considers there to be no merit to an RFC. This should be a last recourse. Instead, suggestions in a “Major @@ -214,8 +335,8 @@ Possible recommendations from **Reviewers** in ascending order of support are: * “Minor changes” suggests that if the described changes are made, that **Editors** can move forward with an RFC without a further review. * “Accept” is a positive vote and no text review is strictly necessary, though - may be provided to add context to the written record. This is equivalent to - the **Reviewer** joining the list of endorsements. + may be provided to add context to the written record. A **Reviewer** who accepts + an RFC is joining the list of endorsements. Three additional versions of the "Accept" recommendation are available for **Reviewers** who additionally maintain an implementation of the NGFF @@ -225,9 +346,11 @@ specification to express further support: * “Implementation complete” with a link to the available code Where a review is required, **Reviewers** are free to structure the text in the -most useful way. There is no template but useful sections might include: +most useful way. A [template markdown file](templates/review_template) +is available but not mandatory. Useful sections include: * Summary +* Conflicts of interest (if they exist) * Significant comments and questions * Minor comments and questions * Recommendation @@ -242,15 +365,17 @@ via subsequent PRs. The response SHOULD include a written rebuttal to each of the reviews. **Editors** include the response in the repository (R5) and contact **Reviewers** to see if their recommendations have changed. -> 🕑 Reviewers responses should be returned in less than two weeks. +> 🕑 Authors responses to Reviewers should be returned to the Editors in less than two weeks. + +(anchor-rebuttal-r6)= This brings a critical, and possibly iterative, decision point (R6). If all **Reviewers** `approve` and there are no further changes needed, the RFC can progress to S1 as soon as there are two in-progress implementations. If the **Reviewers** do _not_ approve, then the **Editors** will make one of three decisions (R7): -This brings a critical, and iterative, decision point (R6). If a "Reject" -recommendation remains, then the RFC is closed. The text remains on the -specification pages for posterity. If sufficient endorsements, -including two in-progress implementations, are available, then the RFC can -progress (S1) to the SPEC phase below. If there are no "Major" objections but -still no consensus, the decision falls to the **Editors** (R7) who may also -move the RFC to the SPEC phase (S0). +* The **Editors** MAY provide **Authors** a list of necessary changes. These will be based on the **Reviewers** suggestions but possibly modified, e.g., to remove contradictions. +* The **Editors** MAY decide that the RFC is to be closed (R9). This is the decision that SHOULD be chosen if there is a unanimous `Reject` recommendation. The **Authors** MAY then decide to re-draft a new RFC (D2). +* Finally, the **Editors** MAY decide that no further changes are necessary (S0). + +If the **Editors** decide to override the recommendations of the **Reviewers** (R7) the **Editors** MUST include a response (S0). This may occur, for example, if consent between the reviewers cannot be reached. In the case of a unanimous `Reject`, the **Editors** SHOULD attempt to find at least one additional, approving **Reviewer** . + +If the **Editors** decide that the changes requested of the **Authors** in R8 are minor, then no further review is necessary and the **Editors** will review any final modifications. (R7) > 📂 Two in-progress implementations required for progressing to S1. @@ -259,6 +384,10 @@ the **Authors** are significant, **Reviewers** may be asked to respond again (R2). Alternatively, **Editors** may send the text back to the **Authors** for further refinement in order to achieve sufficient endorsement. +In all cases, the RFC text and all reviews and comments remain on the specification page for posterity. + +If at any stage in the process above, **Reviewers** feel it necessary to raise an objection to the decisions by the **Editors**, then the “Handling Disagreements” section below should be followed. + Throughout the RFC phase, non-**Author** supporters of the RFC are invited to contact the **Editors**, comment on the original PR, and/or open a new PR to list themselves as **Endorsers** of the RFC. Whether or not the @@ -268,19 +397,23 @@ to contact the **Endorser** regarding any breaking changes. #### SPEC +**Purpose**: During the SPEC phase, necessary changes to the specification and implementations are applied so that the RFC can be officially adopted. + If an RFC enters the SPEC state via **Editors** approval (S0), an additional explanation by the **Editors** will be included in the RFC's directory at which point it is considered equivalent to a **Reviewer** accepted RFC. At this point (S1), the primary purpose of the RFC becomes driving implementations. Further clarifications (S2) may be needed. Updates to the RFC as well as the specification itself will be managed by the **Authors** and the **Editors** in -coordination.**Editors** will also contact remaining implementers (S3) -regarding the status of their implementations and update the endorsements table -accordingly. +coordination. > 🕑 This period may take substantial time to coordinate with implementers, > but should be limited to approximately a month if possible. +**Editors** will also contact remaining implementers (S3) +regarding the status of their implementations and update the endorsements table +accordingly. + Once sufficient endorsements, including two released implementations, are listed, the specification will be considered "adopted". The adopted specification will be slotted into a release version by the **Editors** and the @@ -288,17 +421,124 @@ specification will be slotted into a release version by the **Editors** and the > 📂 Two released implementations required for being adopted. +## Policies + +This section defines several concrete aspects of the RFC process not directly related to the state diagram above. These policies are referred to by section title elsewhere in the document for clarification. They also provide extension points in RFC-1 which may later be amended by separate RFCs. If so, those sections will be marked as deprecated and a reference made to the new RFC. + ### Decision-making Unless otherwise specified in the text, the following considerations are taken into account when making decisions regarding RFCs: + - **prefer working examples**: whether an implementation of an RFC or a failing test which exposes an issue in a proposal, working examples will tend to carry more weight in decision making. + - **technical expertise**: all other considerations being equal, feedback from stakeholders with more technical expertise in a matter under consideration will tend to carry more weight in decision making. +- **newcomer advantage**: care will be taken not to let existing implementations overly dictate the future strategic direction of NGFF in order to avoid premature calcification. + +### RFC Prioritization + +RFCs are welcome from the community as a way to contribute to the public record of NGFF. It is **not** a goal to explicitly limit the number of RFCs and they need not only include specification changes. (See [this list of RFCs](https://en.wikipedia.org/wiki/April_Fools%27_Day_Request_for_Comments) for comparison.) + +That being said, not every RFC MUST be sent to **Reviewers**. Priority will be given to RFCs which move the specification forward followed by RFCs which address non-specification concerns of the community like the process itself. + +### Choice of Reviewers + +Where possible, **Reviewers** SHOULD be chosen to represent a cross-section of +the community. Which cross-sections are chosen MAY depend on a given RFC but +might include geographic distributions, the variety of imaging modalities, +and/or programming languages of the expected implementations. An attempt MUST +also be made to select both supporting and dissenting voices from the community. +*Editors* and *Reviewers* should proactively disclose any potential conflicts +of interest to ensure a transparent review process. + +### Deadline enforcement + +In the absence of concrete mechanisms for deadline enforcement (penalties, etc), all members of the NGFF community and especially the **Editors** SHOULD strive to prevent the specification process from becoming blocked. + +The **Editors**, however will endeavor to: +* keep a record of all communications to identify bottlenecks and improve the RFC process; +* frequently contact **Authors** and **Reviewers** regarding approaching deadlines; +* find new **Reviewers** when it becomes clear that the current slate is overextended; +* and proactively mark RFCs as inactive if it becomes clear that progress has stalled. + +**Authors** and **Reviewers** are encouraged to be open and honest, both with themselves and the other members of the process, on available time. A short message stating that an edit or a review will not occur on deadline or even at all is preferable to silence. + +### Implementation requirements + +The process description describes “sufficient endorsement” in two locations, once for when an RFC can move into the [SPEC phase](#spec) and once when it can be considered “adopted”. This policy describes the current definition of “sufficient”, which is currently kept minimal with an eye towards updating it in the future. + +Under RFC-0, three implementation languages — Javascript, Python, and Java — were considered “reference”, or “required”, for a specification to be complete. This proved a difficult barrier since the implementation teams were not directly funded for work on NGFF. + +RFC-1 has chosen to start with a simpler requirement: **two** separate implementations MUST be _begun_ to enter the SPEC phase and **two** separate implementations (they need not be the same ones) MUST be _released_ to be considered adopted. In both cases, at least **one** of those implementations MUST come from an **Implementer** who is not among the **Authors**. Additionally, data written by both implementations MUST be readable (and therefore validatable) by at least **one** of the implementations. + +It is also strongly encouraged that for each specification change, the [ome-ngff-validator](https://github.com/ome/ome-ngff-validator) additionally be updated. The validator will not fully test the readability of a dataset since it has limited IO capabilities, but it is the most complete tool for validating the metadata associated with a dataset. + +No implementation is currently considered “reference”, though **Editors** may decide that more implementations or a particular are required depending on the complexity of the RFC. + +This policy does not yet specify whether parts of an RFC may be considered _optional_ and therefore omitted from implementations. RFC **Authors** SHOULD provide guidance where possible for **Implementers** on required and optional sections of an RFC until there is a normative policy. + +### RFC Versions + +The IETF RFC process disallows edits to published RFCs. (In the extreme case, a single word change has resulted in a new RFC number.) Though this ensures a unique interpretation of any RFC number, it would also lead to significant duplication of content and _churn_ in the NGFF community. + +Though this decision may be reviewed in the future, RFCs MAY be edited, but **Editors** SHOULD limit modifications to _adopted_ RFCs only for: +- clarification: additional text and examples which simplify the implementation of specifications are welcome; +- deprecation: where sections are no longer accurate and especially when they have been replaced by a new RFC, the existing text can be marked and a link to the updated information provided; +- and extension: references to new RFCs can be added throughout an existing RFC to provide simpler reading for **Implementers**. + +In writing RFCs, **Authors** SHOULD attempt to clearly identify sections which may be deprecated or extended in the future. + +Before an RFC is _adopted_ there are a number of versions of an RFC which are produced during the editing and revision process. This RFC does not try to specify how those versions are managed. The **Editors** are encouraged to layout a best practice as described under “Workflow” that simplifies the review process. Possible solutions include: +* using commit numbers version +* making hard-copies of versions under review +* creating a separate repository per RFC +* opening a long-lived “review PR” with a dedicated URL + +### Specification Versions + +The semantics of versioning for the NGFF specification are not yet sufficiently well-defined. There is currently no clear definition of which parts of the specification, if +any, MUST be implemented by software projects. This has been handled in the +Zarr community by adding a "must understand" flag to extensions to warn +implementations that if they see a particular feature has been included, they +should fail rather than show the user incorrect data. Such a mechanism could +go hand in hand with an as-yet unspecified extension mechanism. This would be +an ideal topic for a following RFC, as well as the existence and handling of possible “development” and/or “long-term support” versions. + +### Work Groups + +Currently, there is a single "track" in the [process diagram](diagram.png) through which +all RFCs progress. This can initially be managed by careful choice of the +reviewers. However, as the complexity of review grows, it may become +advantageous to split the set of a reviewers into separate "tracks", such that +specific topic are handling by area experts. The process may also be modified +on a per track basis. For example, an information track which is largely for +training purposes and provides concrete examples to the community could have +an expedited process. A similar model is in use within the IETF community. If there is interest in establishing a dedicated working group, contact the [Editorial board](#editorial-board). + +### Handling Disagreements - * **Prefer working examples**: whether an implementation of an RFC or a - failing test which exposes an issue in a proposal, working examples will - tend to carry more weight in decision making. - * **technical expertise**: all other considerations being equal, feedback - from stakeholders with more technical expertise in a matter under - consideration will tend to carry more weight in decision making. +The OME community is open to everybody and built upon mutual respect. Nevertheless, disagreements do occur. + +All activities within the NGFF community are conducted under the OME [Code of Conduct](https://github.com/ome/.github/blob/master/CODE_OF_CONDUCT.md#when-something-happens). If you feel that your objections are not being considered, please follow the steps outlined under “When Something Happens”. + +### Advisory Board + +This proposal does not try to define higher-level governance and +conflict resolution. By default, as a repository within the "OME" GitHub +organization, the community health processes defined in +[https://github.com/ome/.github](https://github.com/ome/.github) hold for the NGFF process. However, following +this RFC, additional and specific governance mechanisms can be put in place for +the operation of the NGFF community process. This may or may not follow the +model of other RFC-using communities, e.g., IETF's [IAB](https://www.ietf.org/about/groups/iab/) or Fuchsia's +"governing authority". + +### Editorial Board + +The Editorial Board consists of all current **Editors** and can be reached at \. Josh Moore, as the author of this RFC, is the sole initial member. + +This proposal does not try to define how **Editors** are chosen. Therefore, the use of "Editors" in this document is intended as a placeholder. A future RFC will define the selection and removal of editors. + +### Editorial Workflow + +This proposal also does not define the day-to-day specifics of how **Editors** work. The [Editorial Board](#editorial-board) is encouraged to publish a separate document, e.g. Best Practice Guide, containing the daily or weekly workflow of managing the NGFF process. This document can be more quickly updated than the RFC and can contain updates to GitHub interactions (use of “draft” status, milestones, issues, etc.) and possible automation (reply bots and similar). ## Drawbacks, risks, alternatives, and unknowns @@ -332,7 +572,7 @@ continue to move forward. Will the process be worth the effort? It's unclear. The fact that several other communities have used variants of the process would imply that it can work. It might be that we will have to update and further customize the process for the -NGFF community, but by having something explicit as in this RFC-0 it will make +NGFF community, but by having something explicit as in this RFC-1 it will make it possible for us to more easily iterate. ## Abandoned ideas @@ -369,186 +609,81 @@ This RFC does not try to define all aspects of the NGFF community process and instead focuses on the most immediate block which covers what is typically thought of as the voting process. By establishing this as a foundation, future RFCs can extend the community process either adding or simplifying structure as -feedback determines. In fact, a few locations in the text are marked as -placeholders and will be updated to reference future RFCs as they are defined. +feedback determines. In fact, sections like [“Policies”](#policies) are explicitly +intended to be updated to reference future RFCs as they are defined. The following items are therefore considered out of scope for the purposes of this RFC but future work is intended to evolve the community process. -### A. Editorial Board - -This proposal does not try to define the set of editors or how they are chosen -and operate. However, with the goal of improving and accelerating the evolution -of the specification, that will clearly be necessary. - -### B. Advisory Board - -Similarly, this proposal does not try to define higher-level governance and -conflict resolution. By default, as a repository within the "OME" GitHub -organization, the community health processes defined in - hold for the NGFF process. However, following -this RFC, additional and specific governance mechanisms can be put in place for -the operation of the NGFF community process. This may or may not follow the -model of other RFC-using communities, e.g., IETF's [IAB][iab] or Fuchsia's -"governing authority". - -### C. Participation requirements - -Membership in the editorial and advisory boards but also as a reviewer -could use a clearer definition of timely participation rules to prevent -anyone individual from blocking the process. - -### D. Multiple tracks - -Currently, there is a single "track" in the [process diagram][dia] through which -all RFCs progress. This can initially be managed by careful choice of the -reviewers. However, possibly as the complexity of review grows, it will become -advantageous to split the set of a reviewers into separate "tracks", such that -specific topic are handling by area experts. The process may also be modified -on a per track basis. For example, an information track which is largely for -training purposes and provides concrete examples to the community could have -an expedited process. - -### E. Versions of specifications and RFCs - -The semantics of versioning for the NGFF specification are not well-defined -enough. Are there development versions ("alpha", "beta")? What should a users -expectations be for such versions? Are some versions more stable than others? -Additionally, are RFCs versioned? Can they be (significantly) updated over -time? - -### F. Required specifications and extensions - -There is currently no clear definition of which parts of the specification, if -any, MUST be implemented by software projects. This has been handled in the -Zarr community by adding a "must understand" flag to extensions to warn -implementations that if they see a particular feature has been included, they -should fail rather than show the user incorrect data. Such a mechanism could -go hand in hand with an as-yet unspecified extension mechanism. This would be -an ideal topic for a following RFC. - -### G. Metadata model - -Like other NGFF specifications, the RFC process has an inherent metadata model -which can be captured and actioned upon within the NGFF repository. A draft -representation of this metadata is presented below: - -``` -{ - "@context": "ngff.openmicroscopy.org/rfc", - "@type": "RFC", - "@id": "####", - // Independent of subsequent RFCs - "title": " - "authors": [ - { - "@type": "http://schema.org/Person", - "@id": ... - } - ], - "status": see enum - "published": TODO: follow some schema.org model? WorkOfArt? - "doi": - "edits": [ - { - "authors": - "date": - "xxx": - } - ], - // May be changed by subsequent RFCs - "obsoletes": [], - "obsoleted-by": [], - "updates": [], - "updated-by":[], -} -``` +## Skipped sections -This model is very close to the original IETF RFC model, but omits the -following keywords: +As a non-technical proposal, this RFC excludes the following optional headers: "Performance", "Backwards +Compatibility", "Security considerations", "Privacy considerations", "Testing", +"UI/UX". -- Format: we have limited RFCs to Markdown -- Stream: in IETF, different streams are responsible for different parts of - the internet infrastructure. This may be introduced in the future. -- Similarly the STD (“Standard track”), BCP (“best community practice”), FYI - (“informational”) designations are not currently used. +## Definitions -The possible values for Status are: -- UNKNOWN -- HISTORIC -- INFORMATIONAL -- BEST CURRENT PRACTICE -- EXPERIMENTAL -- PROPOSED STANDARD -- STANDARD +Definitions for terms used throughout this RFC have been collected below. -### H. Misc +(def-accepted)= +**Accepted** +: Specifies that an RFC has passed review and all implementers should begin + implementation if they have not done so already. -Other possibilities that perhaps do not need an RFC but might be worth -considering: +(def-adopted)= +**Adopted** +: An RFC that has been sufficiently implemented to be considered + as active within the community. -- Authors of RFC should be allowed if not encouraged to post their RFCs - to a preprint server. It might then be possible to use the preprint - servers review functionality for the official reviews. -- Other communities like Zarr have discussed having "fast track" specification - changes (e.g., proposing a new codec) that need not follow the entire - process. This may should be considered for certain types of proposals. +(def-author)= +**Author** +: Individual related in proposing an RFC. See ["Stakeholders"](#stakeholders) -## Examples +(def-comment)= +**Comment** +: Documents that are included with the RFC discussing the pros and + cons of the proposal in a structured way. Comments from reviewers are + additionally referred to as "reviews". -Below is a list of the preceding major decisions within the NGFF community that -have been or will be written up following the RFC template proposed here. +(def-draft)= +**Draft** +: First phase of the RFC process. See the related section under “Implementation \> Phases” -| RFC | Title | Decision | -|-----|-----------------------------------------|---------------------------------------| -| TBD | Original NGFF including consensus model | Organically developed | -| TBD | Labels | Adopted under consensus model | -| TBD | HCS | Adopted under consensus model | -| TBD | Tables | Withdrawn (i.e., "tabled") | -| TBD | Transforms | In progress under the consensus model | +(def-editor)= +**Editor** +: Facilitator of the RFC process. See “Stakeholders” -Other RFCs that will _likely_ be written can be found under the "Future -possibilities" section. +(def-endorsement)= +**Endorsement** +: public statement of support, made first on the forums or issue tracker and then recorded within the RFC itself, intended to increase confidence in a proposal. -## Skipped sections +(def-rfc)= +**RFC** ("Request for Comment") +: A formal proposal following a standardized + template that is made to the NGFF repository. The proposal need not be + accepted to be published online. -This RFC excludes the following optional headers: "Performance", "Backwards -Compatibility", "Security considerations", "Privacy considerations", "Testing", -"UI/UX". +(def-pr)= +**PR** +: A pull request opened against the ome/ngff repository on GitHub. -## Definitions +(def-review)= +**Review** +: See Comment -Definitions for terms used throughout this RFC have been collected below. +(def-reviewer)= +**Reviewer** +: Community member invited to attach a more in-depth comment to an RFC. See “Stakeholders” -* Accepted: Specifies that an RFC has passed review and all - implementers should begin implementation if they have not done so already. -* Adopted: An RFC that has been sufficiently implemented to be considered - as active within the community. -* Author: See “Stakeholders” -* Comment: Documents that are included with the RFC discussing the pros and - cons of the proposal in a structured way. Comments from reviewers are - additionally referred to as "reviews". -* Draft: See the related section under “Implementation > Phases” -* Editor: See “Stakeholders” -* Endorsement: -* RFC ("Request for Comment"): A formal proposal following a standardized - template that is made to the NGFF repository. The proposal need not be - accepted to be published online. -* PR: A pull request opened against the ome/ngff repository on GitHub. -* Review: See Comment -* Reviewer: See “Stakeholders” -* ZEP ("Zarr Enhancement Proposal") Decision-making process for the Zarr - specification +(def-zep)= +**ZEP ("Zarr Enhancement Proposal")** +: Decision-making process for the Zarr specification ## Changelog -| Date | Description | Link | -| ---------- | --------------------------------------- | --------------------------------------- | -| 2024-04-24 | Apply changes from comment 1 | | - -[dia]: diagram.png -[tmpl]: template.md -[iab]: https://www.ietf.org/about/groups/iab/ -[1]: https://www.rfc-editor.org/ -[2]: https://github.com/rust-lang/rfcs/blob/master/0000-template.md -[3]: https://works.hashicorp.com/articles/rfc-template -[4]: https://github.com/tensorflow/community/blob/master/rfcs/yyyymmdd-rfc-template.md +| Date | Description | Link | +| ---------- | ---------------------------- | ---------------------------------------------------------------------------- | +| 2024-04-24 | Apply changes from comment 1 | [https://github.com/ome/ngff/pull/231](https://github.com/ome/ngff/pull/231) | +| 2024-08-30 | Apply changes from reviewers | [https://github.com/ome/ngff/pull/258](https://github.com/ome/ngff/pull/258) | +| 2024-10-07 | Apply changes from reviewers | [https://github.com/ome/ngff/pull/263](https://github.com/ome/ngff/pull/263) | +| 2024-10-10 | Apply changes from reviewers | [https://github.com/ome/ngff/pull/266](https://github.com/ome/ngff/pull/266) | +| 2024-10-25 | Adopted | | diff --git a/rfc/1/responses/1/index.md b/rfc/1/responses/1/index.md new file mode 100644 index 00000000..b8816b9c --- /dev/null +++ b/rfc/1/responses/1/index.md @@ -0,0 +1,325 @@ +# RFC-1: Response 1 (2024-04-24 version) + +Many thanks to all of the reviewers of this first RFC. Creating a process like this in isolation is difficult and having your feedback is invaluable. In this response, I’ll begin with overall thoughts covering the entire process. Then per review, I’ll break the review texts into parts, respond to each in turn and point to the changes made to the final text to address raised issues. + +## Overall + +The concrete suggestions as well as the identification of poorly explained or undefined situations have been invaluable and I have endeavored to integrate them into the RFC text. In that process, however, I’ve found that many of the more difficult questions can only be addressed summarily. A theme that arises several times below is that it may not be possible or necessary to specify everything at this point (e.g., the definition of new editors) but I’ve included explicit sections in the updated RFC where future RFCs can “amend” the current proposal. If/when that occurs, the text of the RFC-1 will be updated to point to the new definitions as necessary stating that the new decisions replace, or overrule, the previous sections. In that way, I would hope to find a balance in the response below between answering the most glaring issues and moving the community forward with at least an initial publicly defined process that we can then build on later. + +The major changes/clarifications that I’ve integrated into the text are: +* rejections +* endorsements +* versions +* implementation requirements + +Additionally, all Reviewers were interested in a high-level summary which I’ve added in the RFC-1 [“Proposal”](../../index.md#proposal) overview. Here, especially, a re-review would be appreciated. + +There were a few other high-level feedbacks that I would like to address before beginning the point-by-point response below: + +Firstly, there were a few mentions of “getting stuck”. These included positive examples of where RFC-1 is likely to help prevent NGFF from stagnating, but it also includes questions regarding how RFC-1 will prevent, e.g., a Reviewer not responding. In general, my impression is that no process can completely save us from the limited time and capacity that we have for involvement in the community. In order to achieve a true common specification, it will be necessary for us to find ways to dedicate more time and ultimately to acquire more funding to make it happen. I do not think we must put the entire burden on a single RFC to correct the situation, but I very much believe that having had the RFC-1 document _earlier_ would have significantly helped me to keep the community moving forward. That may be the best we can ask for at the moment, but I very much understand and support the need to continually evaluate and improve the community process moving forward. + +Secondly, there was a mention that the version of the specification (0.4) does not warrant protection from breaking changes, etc. That is certainly the interpretation of an 0.x version under [semver](https://semver.org/), but that has not been formally adopted. Additionally, there are enough members of the community who are actively depending on the stability of the NGFF specification that it is in our best interest to err on the side of caution. Moving forward, an RFC specifically outlining our versioning practice and the definitions for what constitutes a breaking change would be very much welcome. + +Finally, as I’ve mentioned during various discussions, much of the impetus behind RFC-1 is in order to help me keep moving NGFF forward. In fact, since there is no agreed upon process, I am currently working as if RFC-1 is already active to deal with current community suggestions. As a result, there are now several additional RFCs, hopefully a testament to the groundwork here. I respectfully am not convinced by the half-rejection review that we should go back to the drawing board. I appreciate that not everyone had their full say in defining the process, but alternatively, I very much feared how we would make a decision between all of the differing opinions without some initial process in place. This may simply be a case where bootstrapping is needed. I have focused in this rebuttal on reaching a process that we can begin working with and I would hope to identify what that would take for all reviewers and, then, iteratively improve over time. + +Otherwise, please find section by section comments below. +With thanks, +Josh Moore + +## Review 1 + +[Review 1](https://ngff.openmicroscopy.org/rfc/1/review_1.html) was provided by representatives of Swiss Institutions, Kevin Yamauchi, Joel Lüthi, and Virginie Uhlmann. Kevin’s involvement in the tables specification led to many conversations on how the process could be improved. + +### Clarification of progression to SPEC +> Before approving, we would like further clarification on how the key decision points (e.g., R6 and R7) work. In particular, we seek clarification on the decision making powers of the Reviewers and Editors, as well as on the role of Endorsers. … We think there needs to be further clarification on how RFCs transition to the SPEC phase. + +I appreciate the many specific questions. This is definitely one of the interactions which is most at risk of causing conflict. I’ll answer each of your questions briefly here, and then follow with a link to the text that I would add to the RFC. + +> when (can) the Editor … override the Reviewers and vice versa. … we think that the final decision should rest with the Editor. + +I agree that the Editor should be able to override the Reviewers and not vice versa. However, if all the Reviewers `reject` a proposal, there should be some process required of the Editor before accepting the proposal (if ever). If Reviewers are unhappy with an overriding, then they are welcome to add another comment (within the current process). A future RFC should likely define a mechanism for raising official issues or complaints (e.g., see [https://www.ietf.org/contact/](https://www.ietf.org/contact/)) + +> Can the Reviewers “force” an RFC to be accepted in spite of Editor concerns? If so, what are the conditions? + +As RFC-1 currently stands, no. And I do not know if we have sufficient experience to define such a process. I would err on the side of saying that this is an extension that can be spelled out at a future date. + +> It says that “If sufficient endorsements, including two in-progress implementations, are available, then the RFC can progress”, but it isn’t defined what “sufficient endorsements” means. We would appreciate additional clarification. + +This is a very fair question. The difficulty is, of course, that it changes depending on context and is therefore ultimately subjective. The wording I’ve proposed below tries to convey that, e.g., sufficient is _in the opinion of the Editors based on the size of the RFC_. + +> In the current RFC, it seems like a single “Reject” from a reviewer (veto) would prevent the SPEC from being accepted. We think that it is important that an individual cannot block an RFC and that the Editor can override a single “no vote”. As such, we would be in favor of removing the proposed veto power attributed to Reviewers in the current version. + +Thank you for pointing this lack of clarity out. It is certainly not intended that a single reject vetoes. I’ve clarified the text and updated the figure. + +> Similarly, the current text suggests a single “Major changes” recommendation would always send an RFC into another round of edits and review. We think that it’s important that the Editor has some discretion as to whether a “Major changes” recommendation should block an RFC from getting to the SPEC phase. + +As above, I’ve clarified the text to make clear the difference between unanimous “Major changes” and a single recommendation. + + +#### Suggested modified text +Please see [the section on R6 under RFC beginning with “This brings a critical and possibly iterative, decision point](../../index.md#anchor-rebuttal-r6). + +### Clarification on the role of Endorsers +> What is the role of Endorsers in the process? From the diagram, we thought that Endorsers are voting to transition the DRAFT to a SPEC, but it is unclear how that interacts with the rest of the review process (e.g., can Endorsers “override” Reviewers somehow?). However, it is stated “Accept” at R2 is “equivalent to the Reviewer joining the list of endorsements.” and we aren’t sure what that means. We think it’s important that a potential reviewer can endorse a draft be made into an RFC without also simultaneously advocating the RFC should transition to SPEC. + +In general, endorsement is not intended as a core decision-making mechanism, but rather as a trust-building mechanism in order to drive implementations and adoption. I’ve removed the endorsement statement from the text suggestion as well as from the diagram and attempted to overhaul the further description of Endorsement and the templates to make this clear. + + +### The role of Implementers in the process +> We think there needs to be further clarification on the role of the Implementers in the process. What is the intent of the additional version of “accept” for reviewers who are also Implementers (e.g., “Plan to implement”)? Does this mean that Implementers must always be included in the review? Does an RFC need planned implementations to transition to SPEC? It is unclear how these additional types of “Accept” factor into the decision to transition to SPEC. + +These are good questions. Currently, there is a hard-requirement that some implementations be begun in order for the RFC to transition into the SPEC phase. The underlying issue that I foresaw was requiring an **Implementer** to always be chosen as a **Reviewer** because of the limited number of **Implementers** who already have significant work to do. However, this section obviously needs clarification. My proposal would be: + +* An Implementation counts as a Review and therefore can follow the same flow chart without adding any additional routes or new complexity. +* We make clear that there must be at least two Implementation-based Reviews, as the note does in the updated v3.0 diagram. +* We delineate these separate types of feedback — Implementation, Review, Endorsement — in the status table. + +> Further, what happens if a SPEC is accepted but there are no implementations in one of the main “official languages” (i.e., Python, Java, Javascript)? + +Thank you for pointing out how this is currently unspecified. Other terms for what you are describing would include “required implementations” or even “reference implementation(s)”. I’ve suggested adding in another decision node in the graph “Sufficient implementation support” with this specified as an extension point. I have not seen an issue to date such that the “official languages” wouldn’t be involved in implementing an RFC which had made it to this level, but we should plan for that eventuality. At the moment, I’m inclined to leave the requirement at “two released implementations” but we can further specify it down the line. + +### Modification of RFCs and SPECs +> We are curious about how changes to RFCs and SPECs are handled. Are the RFCs versioned as they are updated during the review process (R2-R6)? Is it possible to modify the RFC once it transitions to SPEC? How are changes proposed during the SPEC phase integrated (e.g., are they a PR with changes to the specification)? Do all changes to the spec need to be mentioned in the RFC itself or can finer details just be worked out during the SPEC phase based on a higher-level RFC? For example, if during the implementation, we realize that something needs to be added/changed to the spec, does there need to be another RFC? How is it decided if a change is “big enough” to require a new RFC? We think it should be possible to make minor changes without a new RFC and that the Editor should be able to decide when the magnitude of the change is large enough to warrant a new RFC. + +The RFCs are not currently versioned during the review process (R2-R6) but this is an issue that I have already run into myself. A comment was received with minor improvement suggestions which I implemented while waiting on the reviews, leading Reviewers to rightly point out that they had not taken those changes into account. I’m inclined to create a copy of the RFC accessible from the webpage for each round of reviews to show the major versions over time (e.g., [2024-04-24](../../versions/2024-04-24/index.md)), but I would appreciate further feedback on this point and/or assistance in crafting the versioning RFC to follow. + +### Minor comments and questions + +> It is currently unclear if there are any guidelines or rules for how reviewers are selected in R1. Is there a min/max number of reviewers? On which basis should the reviewers be chosen (eg, Implementers or not)? We would suggest 3-5 would be a reasonable range. + +Thank you for the suggestion. I’ve included it in an extra section which can be amended in the future in case more formalism is required, e.g., spreading over implementation languages, global regions, or imaging modalities. + +> It is currently unclear how reviews are aggregated in R3. Does the Editor just forward all reviews? Or is there some compilation and prioritization provided by the Editor? + +This has been clarified by explaining that the Editors can provide guidance to the Authors such that some sections can be omitted, e.g., where the Reviewers disagree between themselves. + + +> We think it would be helpful to have an explanation of how conflicts of interests are managed and surfaced during the review process. For example, what would be considered a conflict of interest? Are there conflicts of interests that would prevent community members from participating in the review process? + +This is an interesting point. I have updated the template for **Reviewers** to include a section on conflicts and included text in the section on choice of **Reviewers** as well. One-sided situations should be avoided which limit the voices heard. + +However, I would argue that this _wouldn’t_ be a reason to prevent someone from including a review, but it might prompt the **Editors** to look for an _additional_ **Reviewer**. The logic behind this is that a **Review** then functions as a more formal endorsement. + +> Under SPEC it says “Once sufficient endorsements, including two released implementations, are listed, the specification will be considered “adopted”. Is this two implementations in any language (e.g., two Python implementations) or does it have to be implementations in at least two different languages? + +As mentioned above, this has been encapsulated in a new section. I would tend away from being overly proscriptive at this point. The goal is largely to have verified that a specification has been looked at carefully by implementers in order to prevent unintended consequences. + +> We think it would be helpful to have an overview at the top of the RFC giving a high level view of the process. For example, explaining that the proposal transitions from DRAFT -\> RFC -\> SPEC and explaining the intention/goal of each step as well as the estimated time for each (e.g., using the min/max bounds). It might be nice to also have a linear timeline type diagram as well. The detailed diagram in the current RFC is useful for digging into the process, but we find it difficult to quickly understand the overall flow from it. + +Please see the discussion of the summary section at the top level of this document. I very much appreciate, however, the idea of a simplified (and close to linear) diagram for this summary section. It is now included in the document. + +> There are some typos in the document that should be addressed. + +Thank you for pointing these out. They’ve been corrected in the text. + +---- +## Review 2 + +[Review 2](https://ngff--248.org.readthedocs.build/rfc/2/review_2.html) was provided by a collection of members of the Janelia community: Davis Bennett, John Bogovic, Michael Innerberger, Mark Kittisopikul, Virginia Scarlett, Yurii Zubov, some of whom had previously expressed concerns over the process. + +> Please note that the reviewers had mixed opinions on whether a collective review was appropriate. The reviewers proceeded as a courtesy to the author, but clarification on this point is urgently needed. + +Many thanks for attempting the experiment of providing a combined review for individuals at the institution. I apologize if that was more difficult than it needed to be. I would offer that if there is interest in a follow-up review to split it as needed, but I would suggest minimally a separate review per overall recommendation. + +### Review Abstract +> The reviewers commend the author for proposing a well-researched solution to an important and complex set of challenges that have arisen within the OME-NGFF project. The reviewers are particularly pleased with the inclusion of deadlines, which will help changes move forward in a timely manner. However, the reviewers feel that the proposal needs major changes. It is not the reviewers’ intention to antagonize the author, but rather to give OME-NGFF stakeholders another chance to go back to the drawing board together. Including contributors and implementers in the early brainstorming process will help OME-NGFF achieve a decision-making framework that adds the right amount of structure and has broad community support. + +There was and is no impression that any antagonism was intended. I very much appreciate that all the Reviewers have the best interests of the OME-NGFF community in mind and would like to see a vibrant and flourishing process which develops quickly. It’s also critical that your concerns are heard. + +However, though there was perhaps not 100% representation in the lead up to RFC-1, I respectfully disagree that there were insufficient stakeholders involved in the preparation of this process. Additionally, as in the introduction above, even were we to go back to the drawing board, it is unclear how we would then proceed to a decision without a process already in place. + +My impression is that there **is** broad community support. But of course, without having the RFC defined, it was difficult to ask for endorsements. I will explicitly request them once this response has been published to get a measure of the community. + +That being said, the state of the RFC-1 after your valuable feedback still need not be the final word on how the OME-NGFF process functions. We can of course still improve and even replace the RFC, but it would be detrimental to the community to not move forward. + + +### Full Review + +> The reviewers are grateful to the author for taking the initiative to craft this proposal, and for soliciting reviewer feedback. The proposed process is detailed but straightforward. It emphasizes the values of accountability and transparency, which are excellent goals. The reviewers agree that the greatest strength of this proposal is that it gives contributors clear expectations through a defined process and offers *formal deadlines* for each stage. It is expected that these deadlines, tied to each of the three RFC phases, will provide Authors with more structured and stable feedback than might be expected from comments on GitHub. + +I appreciate the Reviewers summary and agree with many of these as the targeted goals of the RFC process. + +> Nevertheless, the reviewers felt that the proposal should not move forward unless major changes are made. The reviewers especially wish to emphasize the desire for greater clarity on the existing problems that this RFC is meant to solve, and why the proposed process is the best way to solve them. + +A non-exhaustive list of problems that NGFF RFCs are intended to solve would include: + +- Providing a clear definition of roles and phases of the overall process: there is some overlap with GitHub roles (maintainer, submitter) but other roles such as **Reviewer** do not have a clear mapping and there is no definition of time extents. +- Increasing the visibility of the discussion process beyond GitHub: a substantial portion of the proto-editorial role even during the original multi-scales issue on the Zarr repository involved periodically stopping the discussion and re-summarizing the state of very long, linear discussions. +- Providing **Editors** a mechanism to *prioritize* certain discussions in the community: the GitHub platform is not built with an editorial voice in mind and only provides blocks of users and comment deletions. +- Most importantly, though, when then there is a disagreement, the current RFC clearly places the burden of decision on the **Editors** which though perhaps implicitly true previously, was not explicitly stated. + +I also would not presume to know if this is the optimal possible process for the community, but I am convinced that it is an improvement over the current state. The RFC process seemed to have many of the values that I as Editor wanted to incorporate after having worked on the NGFF specification for a handful of years. + +#### Scope of the Proposed Process + +> Some reviewers were concerned that the process described by this RFC does not adequately address the OME-NGFF community's most pressing concerns. As such, this section is more about the scope of this RFC than its content. + +> Currently, minor changes to the OME-NGFF specification are not being accepted, while major changes are moving slowly, leading to low morale and risk of contributor burn-out. Consider two cases of stalled revisions: + +Thank you for the concrete examples, they were also in mind while writing RFC-1. + +> While a detailed analysis of these case studies is beyond the scope of this review, suffice it to say that the proposed RFC process would not have helped either of them much. The discussion of units would have been too minor to warrant an RFC. + +In the case of units, if we had been using the RFC-1 process, I imagine I as editor would early on have taken the following steps. When the PR was opened, I would have identified it as breaking: it was a small change but required updates in all implementations. I would have asked for clarifications and possibly endorsements/objections. Finding it not to be a straight-forward matter (i.e., not minor), I would have asked for an RFC whose goal would have been convincing those who objected, i.e., building a consensus. That RFC could have been quickly merged along with the reviews to record the decision. Additionally, having not found a consensus by the end of the thread, I would have felt more empowered to have closed it (R9). I’ll note that I have expressed the fact that there was no consensus as the reason that the PR blocked directly to one of the Reviewers. The fact that this was brought up in this review makes me *more confident* that having the RFC in place to record the result publicly would be advantageous. + +> For the table specification, the RFC process would have drastically reduced the time spent on the proposal, which is good. However, what effectively ended the proposal was that one community liked the table specification and another community didn’t. While the proposed RFC process includes mechanisms for rejection of one solution and acceptance of another, it is not clear that this is the best solution when conflict arises between domain experts using similar data structures for distinct applications. + +I agree that it is not clear that this is the best solution. I would argue that the general feeling is that it is a **good** solution and one that the community is willing to begin with. However, to work again through the hypothetical of the table specification under the RFC process, the most important differences in my mind are that: the original proposal would have been merged sooner and had its endorsements recorded while implementations were worked on. Instead, the specification PR was left open during the implementation period making it less clear to community members who were joining the conversation late what the status of the work was. + +I hope these two examples help to show how the RFC process might have helped. + +> Upon reflection, it becomes clear that proposed revisions to the specification can be major or minor, as well as generic or domain-specific, and these two dimensions have important implications for decision-making. The reviewers would like greater clarity on how this process would be applied to different proposals that vary along these two dimensions. Clarity along the major/minor dimension is discussed further in ‘Major Issues’, below. + +This is a generally well received point. Hopefully the added section helps to clarify the differing view point which is apparent between the Author of this RFC and some of the Reviewers. + +> The reviewers recommend that the author and OME-NGFF stakeholders conduct a retrospective exercise to identify the high-priority problems, and then propose the *minimal amount of structure needed to tackle those particular problems*. A record of this retrospective exercise might belong under RFC-0. + +I am under the impression that much of the community simply wants the NGFF specification process to move forward as quickly as possible. After a period of stagnation related to two large specification efforts which have not yet come to fruition, I don’t have the sense that there is interest in an immediate retrospective. I support, however, such a retrospective. If and when the Reviewers or any other community members are interested in taking that process forward, I will be happy to facilitate. + +That being said, I’m unsure that the minimal amount of structure is necessarily the right way forward either. I’ve erred in this RFC on the side of trying to follow a few, well-established and well-known frameworks (RFC & academic publishing) to make the overall process more familiar. We will inevitably need to refine this process, and if those changes tend towards a *simpler* process, let us do that based on gained experience. + +> The reviewers wish to emphasize that while this review contains many criticisms, it is not intended as a reprimand. The reviewers simply feel that a community discussion, presumably conducted through one or more **virtual meetings**, is vital to developing a procedural framework that will satisfy as many OME-NGFF stakeholders as possible. Since this RFC is already at the REVIEW stage, the only recourse the reviewers have to reinvigorate community discussion is to recommend major changes or rejection. + +I apologize that not all of the Reviewers had an opportunity to participate in a community meeting on the process, but I will note that a few did along with other members of the Janelia community who were invited to review this RFC. Maximizing participation in such discussions has also been my preference and represents the current *status quo* for the NGFF community. + +This suggestion from the Reviewers, however, is equivalent to following that previous [RFC-0](../../../0/index.md) process which has led to extended and inconclusive discussions recently. The fact that there are currently disagreements on _this_ change as evidenced by the Reviewers’ own rejection recommendation makes me doubt that the consensus model of RFC-0 would lead to us now to a more timely conclusion. Instead, a large part of what RFC-1 is attempting to codify is that the Editor **MUST** make these determinations and that he (currently) is empowered to do so. If then an alternative proposal or an improvement for the process is supplied, it can be appropriately and efficiently handled. + +> Below, the reviewers outline major and minor critiques. Some of the critiques suggest tweaks to the existing proposal, but the reader should not take these to mean that the reviewers recommend tweaking the proposal. Scrapping the entire proposal may be more appropriate, in which case many of the comments below will be moot. + +I will follow the strategy that I outlined above: since the author is the current _de factor_ editor and is in need of having _some_ process in place even to go back to the community to make decisions, I will focus below on all suggestions and changes which SHOULD be made. And I’m very appreciate for your having included them as a group. I have also attempted to further emphasize the “yes, when” strategy based on which the review was requested. + +#### Major Issues + +> First, it is not clear to the reviewers that the RFC model is appropriate. The RFC process may introduce more bureaucracy than is needed for this relatively small community (compared to the IETF), which is furthermore still in version 0.x, an early phase [generally associated with rapid iteration](https://semver.org/#spec-item-4). The reviewers recommend holding virtual meetings to discuss alternative procedural frameworks. In particular, the reviewers find three (non-mutually exclusive) alternatives compelling\*. These include: (1) developing a group charter as is common for W3C and IETF working groups; (2) simply curating PRs according to a well-defined scope and editor discretion; and (3) distinguishing between core functionalities and domain-specific extensions. These alternatives employ distinct operational procedures, but they are similar in calling for a clear, simple, and explicit direction of OME-NGFF that would allow the group to achieve core goals while permitting future extension. +> Note that there were six reviewers, and there was disagreement among the reviewers about which alternatives would be best. Pursuing a single alternative may not satisfy all reviewers. + +I appreciate that the review was a difficult task. The fact that the six reviewers had difficulty finding a consensus also speaks to the problem with going back to the drawing board. I take it as a given that there is another approach that _would_ work. There is certainly some subjectivity in the choice, representing a certain style or aesthetic of community management, but additionally, the proposal is based on my own research, my experience managing the NGFF specification, and also being involved with specifications for the past two decades. + +In the way of a brief response to each of the alternatives that the Reviewers list: +1. In researching various W3C work groups, the variability didn’t provide a clear roadmap that I felt I could follow here. We as a community could pick any one, but then would need to learn its idiosyncrasies. Nevertheless, there is certainly much we can learn or adopt from this other groups. For example, within the IETF there are separate streams that may be a model for adding future internal structure (like work groups) if ever needed. +2. I appreciate the hope for simplicity of the GitHub model, and perhaps a simple document saying, “the **Editor** has discretion” would have sufficed for the issues I experienced. I, however, don’t think I would have felt comfortable with that situation. I hope the RFC provides a more balanced mechanism. +3. The distinction between the Core specification and Extensions, also as a mechanism for reducing bureaucracy, is likely a very important part of the NGFF process moving forward. It, however, does not address the decision making that is being sought after by RFC-1. + +> The proposal does not specify how Reviewers and Editors are to be selected. If the Reviewer selection process is arbitrary, then the RFC system does not seem to offer much advantage over the repository owner simply choosing maintainers at their own discretion, as is common for many open-source projects. If the selection process is not arbitrary, then clarity is needed on the criteria for choosing an appropriate Reviewer or Editor, such as the level of interest, investment, and expertise they ought to have in the proposed change, and whether one individual may hold multiple roles. + +This is a very fair point. These were decisions that I felt it was too difficult to spell out at the moment, but I should have been more explicit about that. I’ve now included explicit sections which can be amended in the future to allow the definition of Editorial boards or required reviewers. These choices might, for example, be based on membership, free or paid, in a future organization. + +> The proposal is vague on the criteria for determining whether a change is big enough to warrant an RFC. As discussed in the Scope section above, distinct procedures might be appropriate for major and minor changes.\* +> Again, there was some disagreement among reviewers. One reviewer countered that at [stage 0.x](https://semver.org/#spec-item-4), no changes are truly major, and therefore no change is big enough to warrant an RFC. + +I’ve discussed the versioning in the Overall response, but I agree that this is a critical part of the overall NGFF process. It has received a new section in the text and as mentioned above, but likely also needs a full RFC in the future. + +> The early iteration and vetting stages, specifically (D4), might benefit from more structure. This structure should clarify authors’ responsibilities in relation to comments. For example: +> - Specifying a very clear cut-off, either temporal and/or in terms of scope, between comments the author must address and comments they can safely ignore. +> - Specifying how many endorsements are needed, as well as how to record a non-endorsement (objection). Clarifying the implications of a certain ratio of endorsements to objections, or objections of a certain kind. +> - Offering a mechanism for calling a meeting if the GitHub discussion becomes unwieldy. + +Thank you for the suggestions. I’ve added a section under “Implementation” to clarify this decision node. + +> Explanation is needed as to why ‘Reject’ decisions "should be a last recourse," and what steps will be taken to ensure that a PR would likely be accepted if it becomes an RFC. + +Apologies for leaving that unclear. The choice of making reject a last recourse is fairly common in the RFC community, a value which in part led to it being selected. I’ll add a brief explanation here but add a section in the document with more information. Background reading is available at [https://engineering.squarespace.com/blog/2019/the-power-of-yes-if](https://engineering.squarespace.com/blog/2019/the-power-of-yes-if), etc. Essentially, if responses are codified as “here are the changes that would make me accept this proposal” then the Authors are given a clear path forward rather than just being told no. + +> How are deadlines enforced? What is the protocol for extending deadlines in case of emergency, and/or non-emergency? Clearer policies are needed here. + +These are very good questions. Enforcement in such a volunteer organization is difficult. I think instead that the deadlines should be thought of as giving a guideline to those involved in the process but also the general community on how long the overall process _could_ last. Most concretely, however, the deadlines give the **Editors** a clear remit to get in touch with **Authors** and **Reviewers** and check the status. I don’t believe that we yet need to be proscriptive, e.g., “If a review is not provided by X, then Y will occur.” I’ve included a section under Policies to discuss this. + +> The topics of versioning and extensions are avoided. If these issues are not resolved now, then when? + +Versioning and extension of at least this RFC have now been included. You can find more details under the Overall response at the beginning of this document. The definition of overall extensions of the specification requires, I believe, an RFC itself which is one reason why it would be good to get this process in place as soon as possible. + +#### Minor Issues + +> The reviewers are pleased that implementers are given special attention as Reviewers. However, care should be taken to not let pre-existing implementations dictate the strategic direction of NGFF. + +Thank you for the point. I’ve updated the “Decision-making” section to include this as a general precept. + +> Can the ‘SPEC’ phase simply be eliminated? This is not so much a phase as a one-time event. + +My experience from other specifications is that there are enough steps that take place during this phase that it is useful to be able to point to specifics of what need happen, e.g., the final check of existing implementations as suggested by Reviewer 1. I don’t believe that including it adds so much complexity that it is a detriment, and have updated the diagram to include these additional steps. + +> What is the purpose of the metadata model in (G)? Is this something that might be implemented later? If so, its inclusion at this stage seems premature. + +This is a fair point. Largely, it was included to see if it would resonate with anyone reading the early draft and/or reviewing. That has not been the case to date. I’ve removed the section and it can be resurrected in the future. + +> Publishing reviews on preprint servers may not be a good idea. For one thing, this would substantially scatter communication about RFCs. + +Thank you for the feedback. I would agree that we should *also* include the text in the main repository so that everything is in a single location. But, _if_ anyone in the community is interested in publishing their RFC on a preprint server, I would support it. Similarly, _if_ a review were added to that preprint, I believe the **Editors** would not be overly burdened to copy that review back to the NGFF repository. + +> The Author might consider making the ‘Implementation’ and ‘Tutorials and Examples’ sections required. + +Thank you for the suggestion. I’ve made the Tutorials/Example section “Recommended”. None of the subsections are currently “Required”. I would err on the side of not introducing that at this point, but I take the feedback on board as Editor that I SHOULD strongly urge all Authors to include them. + +#### Writing Style Considerations + +> The template has many sections that seem irrelevant, e.g. ‘Security Considerations’ and ‘Typeface’. + +I have reviewed the section. Certainly, “Typeface” is unnecessary since we are using markdown. Other headers, however, I think serve to encourage Authors to consider many different aspects. (A potential security concern, for example, might arise in an RFC which suggests shipping dynamic code with a dataset for processing.) I’ve attempt to make this more explicit in a new [“Additional considerations”](../../templates/rfc_template.md#additional-considerations) section. + +> In the diagram, only ‘SPEC adopted’ is a proper leaf of the graph. ‘RFC persists’ and ‘PR closed’ seem to feed back to D2 and D3, respectively, so that an infinite loop is created. + +My apologies. I think this is largely a limitation of the drawing software (Google Drawing). There are two arrows that leave the decision nodes (D5 and R7). Depending on which decision was made — “edit”, “withdraw”, “re-draft” — either a loop is entered on the end nodes (D6 and R9) are entered. I’ve now introduced curvy lines for one of the two questions on those nodes to differentiate the two decisions. + +> It would help if there were an extremely basic overview of the process at the top of the document—either a paragraph or a graphical abstract. The current diagram is useful, but is too detailed to quickly convey the important parts of the process. + +Please see the Overall response section at the beginning of this document for a proposed overview. + +> Each of the ‘phases’ sections starts with *a description of what happens* in that phase, but the reviewers recommend starting with the *purpose* of that phase.# RFC 1 + +Thank you for the suggestion. I’ve updated each section accordingly. + +---- + +## Review 3 + +[Review 3](https://ngff.openmicroscopy.org/rfc/1/review_3.html)was provided by Matthew Hartley of the European Bioinformatics Institute. As a representative of the BioImage Archive and EMPIAR, the future of the NGFF specification is of particular concern. + + +### Significant comments and questions + +> In the DRAFT phase, it is somewhat unclear how D4 (“Questions raised during PR review?”) is evaluated - comments on PRs can vary in how clear their intent is to be actionable feedback. If the intent is that the editor(s) have the final call here, it would be useful to indicate this. + +Many thanks. As Reviewer 2 pointed out, this section requires more detail. A new subsection has been added. + +> The proposal makes reference to manuscript review (particularly for the REVIEW phase). In this process, time overruns or nonresponsive participants are common. Some further detail on how such cases would be handled and communicated would be valuable. + +Please refer to the new [“Deadline enforcement”](../../index.md#deadline-enforcement) section of the updated RFC. Beyond regular reminders to Reviewers, the primary mechanism currently at our disposal will be finding additional reviewers. For non-responsive Authors, we should be more proactive in marking an RFC as inactive. There’s some question in my mind of whether or not the regular communication with the Reviewers and Authors should be recorded more visibly as an encouragement or perhaps even driven by a platform or bot of some form. Since this may require frequent improvements, I wouldn’t necessarily specify this in the RFC but leave these decisions for a Best Practice guide for the Editors. + +> Both within the REVIEW and SPEC phases, implementations become important (particularly for reaching “adopted” status). However OME-NGFF is a complex specification and in many cases existing implementations already focus on subsets of the spec. We recommend adding some general indication of what would comprise a minimal implementation necessary to enable adoption. + +Please see the new section [“Implementation requirements”](../../index.md#implementation-requirements) where an attempt is made to specify how the evaluation will be made. As elsewhere, future RFCs may extend this definition. In order to label individual _sections_ of the specification as “required”, another RFC will be needed. + + +> Given exit from the SPEC phase is dependent on complete implementations, care should be taken on how to prevent the RFC getting stuck here. + +The point is well taken. As elsewhere, I believe it will be difficult to _prevent_ getting stuck based on capacity issues, but I think more proactive _detection_ of being stuck will go a long way. I’ve attempted to include this in the section on “Deadline enforcement”. + +> The RFC template needs a revision pass, it has a number of components which are very specific to the project from which the template was copied (presumably the Fuchsia project), e.g. explicit mentions of “FIDL source file compatibility”. + +Apologies. This section has been corrected. + +> More broadly, removing some template sections (e.g. security/privacy/UI/UX), would make sense. + +This is also something I considered. I hoped it would be useful to trigger RFC Editors to consider such issues. I’ve tried to clarify the individual sections. + +### Minor comments and questions + +> During the RFC process, “Reviewers responses should be returned in less than two weeks.” I think this means the responses by the authors to the reviewers? Phrasing is unclear. + + Many thanks. I’ve updated the text to say “Authors”. + +> The role of **Commenters** is a little unclear to me at present. In the definition, they are “invited to add their voice” - is this intended to be an active (e.g. authors/editors ask them) or passive process? + +The intended distinction is that Commenters are welcome to add their Comments to the repository as are Reviewers but that their submissions may not be taken into equal regard when making the final decision. I propose updating the definition with the following: “**Commenters** are other members of the community who, though not contacted as **Reviewers**, have provided feedback that they would like added to the official record of the RFC.” + +> In the figure “RFC persists” indicates that the RFC, although not adopted, remains part of the record of communal work on the specification, but “persists” suggests that it remains in an active process somehow which I do not think is the case. + +This is a fair point. I propose updating the text to “RFC remains on the website” which is what “persists” was intended to communicate. + +> The figure legend suggests that start/end states should have identifiers, this isn’t currently the case for end states. + +Thank you for pointing this out. I’ve labelled the end states with the next numerical state for each of the phases. + +> Very nitpicky - text in some of the state boxes in the figure terminate in “.” characters, others don’t. + +I very much appreciate the attention to detail. I have dropped full stops from the upcoming 3.0 version of the diagram. diff --git a/rfc/1/responses/index.md b/rfc/1/responses/index.md new file mode 100644 index 00000000..08ff92f8 --- /dev/null +++ b/rfc/1/responses/index.md @@ -0,0 +1,9 @@ +# Responses + +Responses from the authors of RFC-1: + +```{toctree} +:maxdepth: 1 +:glob: +*/index +``` diff --git a/rfc/1/review_1.md b/rfc/1/review_1.md deleted file mode 100644 index 79868de3..00000000 --- a/rfc/1/review_1.md +++ /dev/null @@ -1,105 +0,0 @@ -# Review of RFC1 - -## Review authors -This review was written by: -- Joel Lüthi -- Virginie Uhlmann -- Kevin Yamauchi - -## Summary -RFC1 proposes a process by which changes to the NGFF project can be proposed, reviewed, and implemented. These changes include both modifications to the NGFF specification and to the governance of the NGFF project. This process has three phases: DRAFT, RFC, and SPEC. In the draft phase, community members can propose changes. If these changes receive sufficient community support via endorsements and Editor approval, they transition to the RFC phase. In the RFC phase, the proposal is reviewed by Reviewers chosen by an Editor. With Reviewer and Editor approval, the proposal transitions to the SPEC stage in which implementation will begin. - -Overall, we are in favor of the proposed changes and believe they are an improvement over the current process. The current NGFF process for adopting new specifications is based on “community consensus”, which is poorly defined, leading to uncertainty in how and when a specification will be approved. Further, this specification aims to provide a mechanism by which disagreements can be reconciled. Finally, this RFC adds much needed clarity about which type of discussion is meant to be prioritized at a given phase of a proposal. - -We reviewed this RFC from the perspective of somebody who might contribute an NGFF specification and somebody who might make a library that depends on the NGFF. Before approving, we would like further clarification on how the key decision points (e.g., R6 and R7) work. In particular, we seek clarification on the decision making powers of the Reviewers and Editors, as well as on the role of Endorsers. - -## Significant comments and questions -### Interaction of Reviewers and Editor in R6 and R7 -We think there needs to be further clarification on how RFCs transition to the SPEC phase. In particular, it is unclear when the Editor can override the Reviewers and vice versa. Similar to the process established in peer-reviewed publishing, we think that the final decision should rest with the Editor. -In which cases can the Editor override the Reviewers? -Can the Reviewers “force” an RFC to be accepted in spite of Editor concerns? If so, what are the conditions? It says that “If sufficient endorsements, including two in-progress implementations, are available, then the RFC can progress”, but it isn’t defined what “sufficient endorsements” means. We would appreciate additional clarification. -In the current RFC, it seems like a single “Reject” from a reviewer (veto) would prevent the SPEC from being accepted. We think that it is important that an individual cannot block an RFC and that the Editor can override a single “no vote”. As such, we would be in favor of removing the proposed veto power attributed to Reviewers in the current version. -Similarly, the current text suggests a single “Major changes” recommendation would always send an RFC into another round of edits and review. We think that it’s important that the Editor has some discretion as to whether a “Major changes” recommendation should block an RFC from getting to the SPEC phase. - -### Clarification on the role of Endorsers -What is the role of Endorsers in the process? From the diagram, we thought that Endorsers are voting to transition the DRAFT to a SPEC, but it is unclear how that interacts with the rest of the review process (e.g., can Endorsers “override” Reviewers somehow?). However, it is stated “Accept” at R2 is “equivalent to the Reviewer joining the list of endorsements.” and we aren’t sure what that means. We think it’s important that a potential reviewer can endorse a draft be made into an RFC without also simultaneously advocating the RFC should transition to SPEC. - -### The role of Implementers in the process -We think there needs to be further clarification on the role of the Implementers in the process. What is the intent of the additional version of “accept” for reviewers who are also Implementers (e.g., “Plan to implement”)? Does this mean that Implementers must always be included in the review? Does an RFC need planned implementations to transition to SPEC? It is unclear how these additional types of “Accept” factor into the decision to transition to SPEC. - -Further, what happens if a SPEC is accepted but there are no implementations in one of the main “official languages” (i.e., Python, Java, Javascript)? - -### Modification of RFCs and SPECs -We are curious about how changes to RFCs and SPECs are handled. Are the RFCs versioned as they are updated during the review process (R2-R6)? Is it possible to modify the RFC once it transitions to SPEC? How are changes proposed during the SPEC phase integrated (e.g., are they a PR with changes to the specification)? Do all changes to the spec need to be mentioned in the RFC itself or can finer details just be worked out during the SPEC phase based on a higher-level RFC? For example, if during the implementation, we realize that something needs to be added/changed to the spec, does there need to be another RFC? How is it decided if a change is “big enough” to require a new RFC? We think it should be possible to make minor changes without a new RFC and that the Editor should be able to decide when the magnitude of the change is large enough to warrant a new RFC. - -## Minor comments and questions - -- It is currently unclear if there are any guidelines or rules for how reviewers are selected in R1. Is there a min/max number of reviewers? On which basis should the reviewers be chosen (eg, Implementers or not)? We would suggest 3-5 would be a reasonable range. -- It is currently unclear how reviews are aggregated in R3. Does the Editor just forward all reviews? Or is there some compilation and prioritization provided by the Editor? -- We think it would be helpful to have an explanation of how conflicts of interests are managed and surfaced during the review process. For example, what would be considered a conflict of interest? Are there conflicts of interests that would prevent community members from participating in the review process? -Under SPEC it says “Once sufficient endorsements, including two released implementations, are listed, the specification will be considered “adopted”. Is this two implementations in any language (e.g., two Python implementations) or does it have to be implementations in at least two different languages? -- We think it would be helpful to have an overview at the top of the RFC giving a high level view of the process. For example, explaining that the proposal transitions from DRAFT -> RFC -> SPEC and explaining the intention/goal of each step as well as the estimated time for each (e.g., using the min/max bounds). It might be nice to also have a linear timeline type diagram as well. The detailed diagram in the current RFC is useful for digging into the process, but we find it difficult to quickly understand the overall flow from it. -- There are some typos in the document that should be addressed, including: - - end of “Proposal”: _further_RFCs. -> _further_ RFCs () - - In “Implementation”: familiar -> familiarize - - Under “SPEC”: Editors in coordination.Editors -> Editors in coordination. Editors (space missing) - - In “Drawbacks, risks, alternatives, and unknowns”, this RFC is referred to as RFC-0 instead of RFC-1 - -## Recommendation -We are very supportive of this RFC and are appreciative of Josh Moore’s effort to make this proposal. We feel that there needs to be additional clarifications before this RFC can transition to the SPEC stage. Thus, we request “Major changes”, but are optimistic that the requested changes are addressable. - -iff --git a/rfc/1/review_2.md b/rfc/1/review_2.md -eleted file mode 100644 -ndex eba1e6f..0000000 --- a/rfc/1/review_2.md -++ /dev/null -@ -1,50 +0,0 @@ -# Review of RFC2 - -## Review authors -This review was written by: -- Joel Lüthi -- Virginie Uhlmann -- Kevin Yamauchi - -## Summary -RFC1 proposes a process by which changes to the NGFF project can be proposed, reviewed, and implemented. These changes include both modifications to the NGFF specification and to the governance of the NGFF project. This process has three phases: DRAFT, RFC, and SPEC. In the draft phase, community members can propose changes. If these changes receive sufficient community support via endorsements and Editor approval, they transition to the RFC phase. In the RFC phase, the proposal is reviewed by Reviewers chosen by an Editor. With Reviewer and Editor approval, the proposal transitions to the SPEC stage in which implementation will begin. - -Overall, we are in favor of the proposed changes and believe they are an improvement over the current process. The current NGFF process for adopting new specifications is based on “community consensus”, which is poorly defined, leading to uncertainty in how and when a specification will be approved. Further, this specification aims to provide a mechanism by which disagreements can be reconciled. Finally, this RFC adds much needed clarity about which type of discussion is meant to be prioritized at a given phase of a proposal. - -We reviewed this RFC from the perspective of somebody who might contribute an NGFF specification and somebody who might make a library that depends on the NGFF. Before approving, we would like further clarification on how the key decision points (e.g., R6 and R7) work. In particular, we seek clarification on the decision making powers of the Reviewers and Editors, as well as on the role of Endorsers. - -## Significant comments and questions -### Interaction of Reviewers and Editor in R6 and R7 -We think there needs to be further clarification on how RFCs transition to the SPEC phase. In particular, it is unclear when the Editor can override the Reviewers and vice versa. Similar to the process established in peer-reviewed publishing, we think that the final decision should rest with the Editor. -In which cases can the Editor override the Reviewers? -Can the Reviewers “force” an RFC to be accepted in spite of Editor concerns? If so, what are the conditions? It says that “If sufficient endorsements, including two in-progress implementations, are available, then the RFC can progress”, but it isn’t defined what “sufficient endorsements” means. We would appreciate additional clarification. -In the current RFC, it seems like a single “Reject” from a reviewer (veto) would prevent the SPEC from being accepted. We think that it is important that an individual cannot block an RFC and that the Editor can override a single “no vote”. As such, we would be in favor of removing the proposed veto power attributed to Reviewers in the current version. -Similarly, the current text suggests a single “Major changes” recommendation would always send an RFC into another round of edits and review. We think that it’s important that the Editor has some discretion as to whether a “Major changes” recommendation should block an RFC from getting to the SPEC phase. - -### Clarification on the role of Endorsers -What is the role of Endorsers in the process? From the diagram, we thought that Endorsers are voting to transition the DRAFT to a SPEC, but it is unclear how that interacts with the rest of the review process (e.g., can Endorsers “override” Reviewers somehow?). However, it is stated “Accept” at R2 is “equivalent to the Reviewer joining the list of endorsements.” and we aren’t sure what that means. We think it’s important that a potential reviewer can endorse a draft be made into an RFC without also simultaneously advocating the RFC should transition to SPEC. - -### The role of Implementers in the process -We think there needs to be further clarification on the role of the Implementers in the process. What is the intent of the additional version of “accept” for reviewers who are also Implementers (e.g., “Plan to implement”)? Does this mean that Implementers must always be included in the review? Does an RFC need planned implementations to transition to SPEC? It is unclear how these additional types of “Accept” factor into the decision to transition to SPEC. - -Further, what happens if a SPEC is accepted but there are no implementations in one of the main “official languages” (i.e., Python, Java, Javascript)? - -### Modification of RFCs and SPECs -We are curious about how changes to RFCs and SPECs are handled. Are the RFCs versioned as they are updated during the review process (R2-R6)? Is it possible to modify the RFC once it transitions to SPEC? How are changes proposed during the SPEC phase integrated (e.g., are they a PR with changes to the specification)? Do all changes to the spec need to be mentioned in the RFC itself or can finer details just be worked out during the SPEC phase based on a higher-level RFC? For example, if during the implementation, we realize that something needs to be added/changed to the spec, does there need to be another RFC? How is it decided if a change is “big enough” to require a new RFC? We think it should be possible to make minor changes without a new RFC and that the Editor should be able to decide when the magnitude of the change is large enough to warrant a new RFC. - -## Minor comments and questions - -- It is currently unclear if there are any guidelines or rules for how reviewers are selected in R1. Is there a min/max number of reviewers? On which basis should the reviewers be chosen (eg, Implementers or not)? We would suggest 3-5 would be a reasonable range. -- It is currently unclear how reviews are aggregated in R3. Does the Editor just forward all reviews? Or is there some compilation and prioritization provided by the Editor? -- We think it would be helpful to have an explanation of how conflicts of interests are managed and surfaced during the review process. For example, what would be considered a conflict of interest? Are there conflicts of interests that would prevent community members from participating in the review process? -Under SPEC it says “Once sufficient endorsements, including two released implementations, are listed, the specification will be considered “adopted”. Is this two implementations in any language (e.g., two Python implementations) or does it have to be implementations in at least two different languages? -- We think it would be helpful to have an overview at the top of the RFC giving a high level view of the process. For example, explaining that the proposal transitions from DRAFT -> RFC -> SPEC and explaining the intention/goal of each step as well as the estimated time for each (e.g., using the min/max bounds). It might be nice to also have a linear timeline type diagram as well. The detailed diagram in the current RFC is useful for digging into the process, but we find it difficult to quickly understand the overall flow from it. -- There are some typos in the document that should be addressed, including: - - end of “Proposal”: _further_RFCs. -> _further_ RFCs () - - In “Implementation”: familiar -> familiarize - - Under “SPEC”: Editors in coordination.Editors -> Editors in coordination. Editors (space missing) - - In “Drawbacks, risks, alternatives, and unknowns”, this RFC is referred to as RFC-0 instead of RFC-1 - -## Recommendation -We are very supportive of this RFC and are appreciative of Josh Moore’s effort to make this proposal. We feel that there needs to be additional clarifications before this RFC can transition to the SPEC stage. Thus, we request “Major changes”, but are optimistic that the requested changes are addressable. diff --git a/rfc/1/reviews/1/index.md b/rfc/1/reviews/1/index.md new file mode 100644 index 00000000..154273eb --- /dev/null +++ b/rfc/1/reviews/1/index.md @@ -0,0 +1,49 @@ +# RFC-1: Review 1 + +## Review authors +This review was written by: +- Joel Lüthi +- Virginie Uhlmann +- Kevin Yamauchi + +## Summary +RFC1 proposes a process by which changes to the NGFF project can be proposed, reviewed, and implemented. These changes include both modifications to the NGFF specification and to the governance of the NGFF project. This process has three phases: DRAFT, RFC, and SPEC. In the draft phase, community members can propose changes. If these changes receive sufficient community support via endorsements and Editor approval, they transition to the RFC phase. In the RFC phase, the proposal is reviewed by Reviewers chosen by an Editor. With Reviewer and Editor approval, the proposal transitions to the SPEC stage in which implementation will begin. + +Overall, we are in favor of the proposed changes and believe they are an improvement over the current process. The current NGFF process for adopting new specifications is based on “community consensus”, which is poorly defined, leading to uncertainty in how and when a specification will be approved. Further, this specification aims to provide a mechanism by which disagreements can be reconciled. Finally, this RFC adds much needed clarity about which type of discussion is meant to be prioritized at a given phase of a proposal. + +We reviewed this RFC from the perspective of somebody who might contribute an NGFF specification and somebody who might make a library that depends on the NGFF. Before approving, we would like further clarification on how the key decision points (e.g., R6 and R7) work. In particular, we seek clarification on the decision making powers of the Reviewers and Editors, as well as on the role of Endorsers. + +## Significant comments and questions +### Interaction of Reviewers and Editor in R6 and R7 +We think there needs to be further clarification on how RFCs transition to the SPEC phase. In particular, it is unclear when the Editor can override the Reviewers and vice versa. Similar to the process established in peer-reviewed publishing, we think that the final decision should rest with the Editor. +In which cases can the Editor override the Reviewers? +Can the Reviewers “force” an RFC to be accepted in spite of Editor concerns? If so, what are the conditions? It says that “If sufficient endorsements, including two in-progress implementations, are available, then the RFC can progress”, but it isn’t defined what “sufficient endorsements” means. We would appreciate additional clarification. +In the current RFC, it seems like a single “Reject” from a reviewer (veto) would prevent the SPEC from being accepted. We think that it is important that an individual cannot block an RFC and that the Editor can override a single “no vote”. As such, we would be in favor of removing the proposed veto power attributed to Reviewers in the current version. +Similarly, the current text suggests a single “Major changes” recommendation would always send an RFC into another round of edits and review. We think that it’s important that the Editor has some discretion as to whether a “Major changes” recommendation should block an RFC from getting to the SPEC phase. + +### Clarification on the role of Endorsers +What is the role of Endorsers in the process? From the diagram, we thought that Endorsers are voting to transition the DRAFT to a SPEC, but it is unclear how that interacts with the rest of the review process (e.g., can Endorsers “override” Reviewers somehow?). However, it is stated “Accept” at R2 is “equivalent to the Reviewer joining the list of endorsements.” and we aren’t sure what that means. We think it’s important that a potential reviewer can endorse a draft be made into an RFC without also simultaneously advocating the RFC should transition to SPEC. + +### The role of Implementers in the process +We think there needs to be further clarification on the role of the Implementers in the process. What is the intent of the additional version of “accept” for reviewers who are also Implementers (e.g., “Plan to implement”)? Does this mean that Implementers must always be included in the review? Does an RFC need planned implementations to transition to SPEC? It is unclear how these additional types of “Accept” factor into the decision to transition to SPEC. + +Further, what happens if a SPEC is accepted but there are no implementations in one of the main “official languages” (i.e., Python, Java, Javascript)? + +### Modification of RFCs and SPECs +We are curious about how changes to RFCs and SPECs are handled. Are the RFCs versioned as they are updated during the review process (R2-R6)? Is it possible to modify the RFC once it transitions to SPEC? How are changes proposed during the SPEC phase integrated (e.g., are they a PR with changes to the specification)? Do all changes to the spec need to be mentioned in the RFC itself or can finer details just be worked out during the SPEC phase based on a higher-level RFC? For example, if during the implementation, we realize that something needs to be added/changed to the spec, does there need to be another RFC? How is it decided if a change is “big enough” to require a new RFC? We think it should be possible to make minor changes without a new RFC and that the Editor should be able to decide when the magnitude of the change is large enough to warrant a new RFC. + +## Minor comments and questions + +- It is currently unclear if there are any guidelines or rules for how reviewers are selected in R1. Is there a min/max number of reviewers? On which basis should the reviewers be chosen (eg, Implementers or not)? We would suggest 3-5 would be a reasonable range. +- It is currently unclear how reviews are aggregated in R3. Does the Editor just forward all reviews? Or is there some compilation and prioritization provided by the Editor? +- We think it would be helpful to have an explanation of how conflicts of interests are managed and surfaced during the review process. For example, what would be considered a conflict of interest? Are there conflicts of interests that would prevent community members from participating in the review process? +Under SPEC it says “Once sufficient endorsements, including two released implementations, are listed, the specification will be considered “adopted”. Is this two implementations in any language (e.g., two Python implementations) or does it have to be implementations in at least two different languages? +- We think it would be helpful to have an overview at the top of the RFC giving a high level view of the process. For example, explaining that the proposal transitions from DRAFT -> RFC -> SPEC and explaining the intention/goal of each step as well as the estimated time for each (e.g., using the min/max bounds). It might be nice to also have a linear timeline type diagram as well. The detailed diagram in the current RFC is useful for digging into the process, but we find it difficult to quickly understand the overall flow from it. +- There are some typos in the document that should be addressed, including: + - end of “Proposal”: _further_RFCs. -> _further_ RFCs () + - In “Implementation”: familiar -> familiarize + - Under “SPEC”: Editors in coordination.Editors -> Editors in coordination. Editors (space missing) + - In “Drawbacks, risks, alternatives, and unknowns”, this RFC is referred to as RFC-0 instead of RFC-1 + +## Recommendation +We are very supportive of this RFC and are appreciative of Josh Moore’s effort to make this proposal. We feel that there needs to be additional clarifications before this RFC can transition to the SPEC stage. Thus, we request “Major changes”, but are optimistic that the requested changes are addressable. diff --git a/rfc/1/reviews/1b/index.md b/rfc/1/reviews/1b/index.md new file mode 100644 index 00000000..c5fc4e44 --- /dev/null +++ b/rfc/1/reviews/1b/index.md @@ -0,0 +1,65 @@ +# RFC-1: Review 1 Round 2 + +## Review authors +This review was written by: +- Joel Lüthi +- Virginie Uhlmann +- Kevin Yamauchi + +## Summary + +We commend the Author for their thorough revision and detailed response to our +comments. The revision has significantly improved the quality of the RFC +overall. We thus recommend the acceptance of this RFC. We are particularly +grateful for the following changes, which greatly enhance the readability and +clarity of the document: + +- The high level diagram is extremely useful and contributes to making the + document more approachable and easier to understand; +- The roles and responsibilities of each party and of the Editor have been + appropriately clarified; +- The purpose statements provided for every phase are incredibly useful and + clearly convey the rationale behind each step of the process. + +As mentioned in our initial review, we feel that the process introduced in this +RFC is essential to allow the community to move forward and iterate upon the +NGFF. Given the importance of such a process, we believe that the proposed +process is viable and that pragmatism should be favored over perfection. As +accurately pointed out in the response, clarity on the NGFF specification +process is critically needed for the continuation and success of NGFF, and the +proposed solution fulfills this need. + +We have read the feedback from Review 2 and we acknowledge that the proposed +process is likely not perfect. We however feel that many of the requests for +clarification have been fully satisfied and that the proposal provides the +opportunity for community contributors who may not be satisfied with the +current process to propose alternatives or amendments by submitting their own +RFC. + +## Minor comments and questions + +1. Definition of Endorsers: While the new definition of endorsers is + commendable, we still have concerns regarding whether a potential reviewer + can endorse an early RFC draft. In the “Reviewer Accept” section, the phrase + “This is equivalent to the Reviewer joining the list of endorsements” might + be interpreted as “If I endorse a draft proposal, my review verdict would be + accept,” suggesting that a review acceptance is equivalent to an + endorsement. To clarify, we recommend rewording this as: “[...] additional + context may be provided for the written record. A Reviewer who accepts an + RFC is joining the list of endorsements.” +2. Conflict of Interest Inclusion: We agree that the goal is not to silence + voices with potential conflicts of interest but to ensure transparency. We + support the addition of a “Conflicts of Interest” section in the review + process. We suggest that RFC1 include “Conflicts of Interest” as a “useful + section to include” in the review and that it be mentioned in the “Choice of + Reviewers” section. A possible wording could be: “Editors and Reviewers + should proactively disclose any potential conflicts of interest to ensure a + transparent review process.” +3. Minor Typographical Note: In the “Implementation” section, it mentions + “various stakeholders (in bold)”; however, in Figure 2, the stakeholders are + no longer highlighted in bold. We recommend updating the figure or adjusting + the text to reflect the current formatting. + +## Recommendation + +We recommend that RFC1 be accepted. diff --git a/rfc/1/review_2.md b/rfc/1/reviews/2/index.md similarity index 99% rename from rfc/1/review_2.md rename to rfc/1/reviews/2/index.md index f3bde5e7..738475ab 100644 --- a/rfc/1/review_2.md +++ b/rfc/1/reviews/2/index.md @@ -1,4 +1,4 @@ -# RFC-1 Review +# RFC-1: Review 2 ## Contributors @@ -88,4 +88,4 @@ Below, the reviewers outline major and minor critiques. Some of the critiques su - The template has many sections that seem irrelevant, e.g. ‘Security Considerations’ and ‘Typeface’. - In the diagram, only ‘SPEC adopted’ is a proper leaf of the graph. ‘RFC persists’ and ‘PR closed’ seem to feed back to D2 and D3, respectively, so that an infinite loop is created. - It would help if there were an extremely basic overview of the process at the top of the document—either a paragraph or a graphical abstract. The current diagram is useful, but is too detailed to quickly convey the important parts of the process. -- Each of the ‘phases’ sections starts with *a description of what happens* in that phase, but the reviewers recommend starting with the *purpose* of that phase. \ No newline at end of file +- Each of the ‘phases’ sections starts with *a description of what happens* in that phase, but the reviewers recommend starting with the *purpose* of that phase. diff --git a/rfc/1/reviews/2b/index.md b/rfc/1/reviews/2b/index.md new file mode 100644 index 00000000..91d57e71 --- /dev/null +++ b/rfc/1/reviews/2b/index.md @@ -0,0 +1,29 @@ +# RFC-1: Review 2b + +## Contributors + +The contributors to this review were John Bogovic, Michael Innerberger, and Virginia Scarlett (referred to in this document as 'the reviewers'). This review does not represent the opinions of any other individuals, nor of HHMI Janelia as an institution. + +## Decision: Accept + +John Bogovic: Accept +Michael Innerberger: Accept +Virginia Scarlett: Accept + +## Decision Statement + +The reviewers appreciate the changes made to improve the proposal, including updates to versioning and decision-making. +While only time will reveal how the process functions in practice, the reviewers agree with the author that moving forward is paramount. + +Given the author's confidence that the RFC process has broad community support and requires no major revisions at this time, the reviewers doubt that it would be fruitful to delay the process further by insisting on their stance. +Most of the present reviewers feel that the proposed review process remains overly complex, and lacks mechanisms to swiftly address divergent recommendations. +Nevertheless, even though the reviewers feel that the proposal still requires major changes, they change their recommendation to 'accept', acknowledging the importance of forward progress. +As the OME-Zarr community moves forward using this proposed process, the reviewers hope that their concerns will be kept in mind and revisited if / when these concerns become practical issues. + +Below are some minor changes the author may consider. If the author chooses not to implement them, the reviewers will not contest that decision. + +## Suggested Minor Changes + +1. Typo: “The RFC process can be represented as a state diagram with the various stakeholders (in ~~bold~~ capital letters) responsible for forward motion.” +2. Regarding this sentence: “An attempt MUST also be made to select both positive and negative voices from the community.” This sentence seems to refer to positive and negative *people*. 'Endorsers' and 'objectors' would be better. +3. Specification Versions, Work Groups, Advisory Board, and Editorial Workflow: These sections describe future possibilities, and so should go in the Future Possibilities section. diff --git a/rfc/1/review_3.md b/rfc/1/reviews/3/index.md similarity index 99% rename from rfc/1/review_3.md rename to rfc/1/reviews/3/index.md index f3a3c7c5..902ffe5f 100644 --- a/rfc/1/review_3.md +++ b/rfc/1/reviews/3/index.md @@ -1,4 +1,4 @@ -# RFC 1 +# RFC-1: Review 3 This review submitted by Matthew Hartley, on behalf on EMBL-EBI's imaging data resources (BioImage Archive, EMPIAR and EMDB). diff --git a/rfc/1/reviews/index.md b/rfc/1/reviews/index.md new file mode 100644 index 00000000..9ae3b6c6 --- /dev/null +++ b/rfc/1/reviews/index.md @@ -0,0 +1,9 @@ +# Reviews + +Reviews of RFC-1: + +```{toctree} +:maxdepth: 1 +:glob: +*/index +``` diff --git a/rfc/1/templates/index.md b/rfc/1/templates/index.md new file mode 100644 index 00000000..e0e1dc2c --- /dev/null +++ b/rfc/1/templates/index.md @@ -0,0 +1,9 @@ +# Templates + +Templates provided for the RFC process: + +```{toctree} +:maxdepth: 1 +rfc_template +review_template +``` diff --git a/rfc/1/templates/review_template.md b/rfc/1/templates/review_template.md new file mode 100644 index 00000000..10597d42 --- /dev/null +++ b/rfc/1/templates/review_template.md @@ -0,0 +1,28 @@ +# Review Template + +Replace the title above of this file with “RFC-NUM: Review NUM” + +## Review authors + +## Conflicts of interest (optional) + +This section should be included if authors feel that there is any background information (shared grants, financial interests, etc.) which should be shared with the community for transparency. This will not prevent the **Review Authors** from submitting a review, but may trigger the addition of other **Reviewers**. + +## Summary + +## Significant comments and questions + +### Subheadings + +Structure any subheadings as necessary. + +## Minor comments and questions + +Similarly, add any subheadings necessary + +## Recommendation + +Adopt, major changes, minor changes, reject (as last resort) + +See [the list of recommendations under “RFC” in RFC-1](../index.md#rfc-recommendations). + diff --git a/rfc/1/templates/rfc_template.md b/rfc/1/templates/rfc_template.md new file mode 100644 index 00000000..a4dfda4c --- /dev/null +++ b/rfc/1/templates/rfc_template.md @@ -0,0 +1,302 @@ +@: template +# RFC Template + +Summary: Sentence fragment summary + +## Status + +Brief description of status, including the state identifier, e.g. `R4` + +| Name | GitHub Handle | Institution | Date | Status | +| --------- | ------------- | ----------- | ---------- | ------------------------------------- | +| Author | N/A | N/A | xxxx-xx-xx | Author | +| Author | N/A | N/A | xxxx-xx-xx | Author; Implemented (link to release) | +| Commenter | N/A | N/A | xxxx-xx-xx | Endorse (link to comment) | +| Commenter | N/A | N/A | xxxx-xx-xx | Not yet (link to comment) | +| Endorser | N/A | N/A | xxxx-xx-xx | Endorse (no link needed) | +| Endorser | N/A | N/A | xxxx-xx-xx | Implementing (link to branch/PR) | +| Reviewer | N/A | N/A | xxxx-xx-xx | Endorse (link to comment) | +| Reviewer | N/A | N/A | xxxx-xx-xx | Requested by editor | + +## Overview + +The RFC begins with a brief overview. This section should be one or two +paragraphs that just explains what the goal of this RFC is going to be, but +without diving too deeply into the "why", "why now", "how", etc. Ensure anyone +opening the document will form a clear understanding of the RFCs intent from +reading this paragraph(s). + +## Background + +The next section is the "Background" section. This section should be at least +two paragraphs and can take up to a whole page in some cases. The \*\*guiding goal +of the background section\*\* is: as a newcomer to this project (new employee, team +transfer), can I read the background section and follow any links to get the +full context of why this change is necessary? + +If you can't show a random engineer the background section and have them +acquire nearly full context on the necessity for the RFC, then the background +section is not full enough. To help achieve this, link to prior RFCs, +discussions, and more here as necessary to provide context so you don't have to +simply repeat yourself. + +## Proposal + +The next required section is "Proposal". Given the background above, this +section proposes a solution. This should be an overview of the "how" for the +solution, but for details further sections will be used. + +## Sections (at heading 2) + +From this point onwards, the sections and headers are generally freeform +depending on the RFC, though it is typically preferable to make use of the +sections listed below changing the order as necessary. Sections are styled as +"Heading 2". Try to organize your information into self-contained sections that +answer some critical question, and organize your sections into an order that +builds up knowledge necessary (rather than forcing a reader to jump around to +gain context). + +Sections often are split further into sub-sections styled "Heading 3" and beyond. These sub-sections just further help to organize data to ease reading and discussion. + +## Requirements (Recommended Header) + +For the problem(s) solved by this RFC, what constrains the possible solutions? +List other RFCs, or standards (ISO, etc.) which are applicable. It is suggested +that the following text SHOULD be used in all RFCs: + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in [IETF RFC 2119](https://tools.ietf.org/html/rfc2119) + +## Stakeholders (Recommended Header) + +Who has a stake in whether this RFC is accepted? + +* Facilitator: The person appointed to shepherd this RFC through the RFC + process. +* Reviewers: List people whose vote (+1 or -1) will be taken into consideration + by the editor when deciding whether this RFC is accepted or rejected. Where + applicable, also list the area they are expected to focus on. In some cases + this section may be initially left blank and stakeholder discovery completed + after an initial round of socialization. Care should be taken to keep the + number of reviewers manageable, although the exact number will depend on the + scope of the RFC in question. +* Consulted: List people who should review the RFC, but whose approval is not + required. +* Socialization: This section may be used to describe how the design was + socialized before advancing to the "Iterate" stage of the RFC process. For + example: "This RFC was discussed at a working group meetings from 20xx-20yy" + +## Implementation (Recommended Header) + +Many RFCs have an "implementation" section which details how the implementation +will work. This section should explain the rough specification changes. The +goal is to give an idea to reviewers about the subsystems that require change +and the surface area of those changes. + +This knowledge can result in recommendations for alternate approaches that +perhaps are idiomatic to the project or result in less packages touched. Or, it +may result in the realization that the proposed solution in this RFC is too +complex given the problem. + +For the RFC author, typing out the implementation in a high-level often serves +as "[rubber duck debugging](https://en.wikipedia.org/wiki/Rubber_duck_debugging)" and you can catch a lot of +issues or unknown unknowns prior to writing any real code. + +## Drawbacks, risks, alternatives, and unknowns (Recommended Header) + +* What are the costs of implementing this proposal? +* What known risks exist? What factors may complicate your project? Include: + security, complexity, compatibility, latency, service immaturity, lack of + team expertise, etc. +* What other strategies might solve the same problem? +* What questions still need to be resolved, or details iterated upon, to accept + this proposal? Your answer to this is likely to evolve as the proposal + evolves. +* What parts of the design do you expect to resolve through the RFC process + before this gets merged? +* What parts of the design do you expect to resolve through the implementation + of this feature before stabilization? +* What related issues do you consider out of scope for this RFC that could be + addressed in the future independently of the solution that comes out of this + RFC? + +## Abandoned Ideas (Optional Header) + +As RFCs evolve, it is common that there are ideas that are abandoned. Rather +than simply deleting them from the document, you should try to organize them +into sections that make it clear they're abandoned while explaining why they +were abandoned. + +When sharing your RFC with others or having someone look back on your RFC in +the future, it is common to walk the same path and fall into the same pitfalls +that we've since matured from. Abandoned ideas are a way to recognize that path +and explain the pitfalls and why they were abandoned. + +## Prior art and references (Optional Header) + +Is there any background material that might be helpful when reading this +proposal? For instance, do other operating systems address the same problem +this proposal addresses? + +Discuss prior art, both the good and the bad, in relation to this proposal. A +few examples of what this can include are: + +Does this feature exist in other formats and what experiences has their +community had? + +Are there any published papers or great posts that discuss this? If you have +some relevant papers to refer to, this can serve as a more detailed theoretical +background. + +This section is intended to encourage you as an author to think about the +lessons from other domains, and provide readers of your RFC with a fuller +picture. If there is no prior art, that is fine - your ideas are interesting to +us whether they are brand new or if it is an adaptation from other languages. + +Note that while precedent set by other languages is some motivation, it does +not on its own motivate an RFC. + +## Future possibilities (Optional Header) + +Think about what the natural extension and evolution of your proposal would be +and how it would affect the specification and project as a whole in a holistic +way. Try to use this section as a tool to more fully consider all possible +interactions with the project in your proposal. Also consider how this all fits +into the roadmap for the project and of the relevant sub-team. + +This is also a good place to "dump ideas", if they are out of scope for the RFC +you are writing but otherwise related. If you have tried and cannot think of +any future possibilities, you may simply state that you cannot think of +anything. + +Note that having something written down in the future-possibilities section is +not a reason to accept the current or a future RFC; such notes should be in the +section on motivation or rationale in this or subsequent RFCs. The section +merely provides additional information. + +## Performance (Recommended Header) + +What impact will this proposal have on performance? What benchmarks should we +create to evaluate the proposal? To evaluate the implementation? Which of those +benchmarks should we monitor on an ongoing basis? + +Do you expect any (speed / memory)? How will you confirm? + +There should be microbenchmarks. Are there? + +There should be end-to-end tests and benchmarks. If there are not (since this +is still a design), how will you track that these will be created? + +## Compatibility (Recommended Header) + +How does this proposal affect backwards and forwards compatibility? + +Does it restrict existing assumptions or remove existing restrictions? + +How are implementations expected to handle these changes? + +## Testing (Recommended Header) + +How will you test your feature? A typical testing strategy involves unit, +integration, and end-to-end tests. Are our existing test frameworks and +infrastructure sufficient to support these tests or does this proposal require +additional investment in those areas? + +If your proposal defines a contract implemented by other people, how will those +people test that they have implemented the contract correctly? Consider, for +example, creating a conformance test suite for this purpose. + +## Tutorials and Examples (Recommended Header) + +It is strongly recommended to provide as many examples as possible of what both users and developers can expect if the RFC were to be accepted. Sample data should be shared publicly. If longer-term is not available, contact the **Editors** for assistance. + +(additional-considerations)= +## Additional considerations (Optional Header) + +Most RFCs will not need to consider all the following issues. They are included here as a checklist + +### Security +What impact will this proposal have on security? Does the proposal require a +security review? + +A good starting point is to think about how the system might encounter +untrusted inputs and how those inputs might be used to manipulate the system. +From there, consider how known classes of vulnerabilities might apply to the +system and what tools and techniques can be applied to avoid those +vulnerabilities. + +### Privacy + +What impact will this proposal have on privacy? Does the proposal require a +privacy review? + +A good starting point is to think about how user data might be collected, +stored, or processed by your system. From there, consider the lifecycle of such +data and any data protection techniques that may be employed. + +### UI/UX + +If there are user- or frontend-impacting changes by this RFC, it is important +to have a "UI/UX" section. User-impacting changes might include changes in how +images will be rendered. Frontend-impacting changes might include the need to +perform additional preprocessing of inputs before displaying to users. + +This section is effectively the "implementation" section for the user +experience. The goal is to explain the changes necessary, any impacts to +backwards compatibility, any impacts to normal workflow, etc. + +As a reviewer, this section should be checked to see if the proposed changes +feel like the rest of the ecosystem. Further, if the breaking changes are +intolerable or there is a way to make a change while preserving compatibility, +that should be explored. + +## Style Notes (EXAMPLE) + +All RFCs should follow similar styling and structure to ease reading. + +This section will updated as more style decisions are made +so that users of the template can simply cut-n-paste sections. + +### Heading Styles + +"Heading 2" should be used for section titles. We do not use "Heading 1" +because aesthetically the text is too large. Google Docs will use Heading 2 as +the outermost headers in the generated outline. + +"Heading 3" should be used for sub-sections. + +Further heading styles can be used for nested sections, however it is rare that +a RFC goes beyond "Heading 4," and rare itself that "Heading 4" is reached. + +### Lists + +When making lists, it is common to bold the first phrase/sentence/word to bring +some category or point to attention. For example, a list of API considerations: + +* *Format* should be widgets +* *Protocol* should be widgets-rpc +* *Backwards* compatibility should be considered. + +### Spelling + +American spelling is preferred. + +### Code Samples + +Code samples should be indented (tab or spaces are fine as long as it is +consistent) text using the Courier New font. Syntax highlighting can be +included if possible but isn't necessary. Please ensure the highlighted syntax +is the proper font size and using the font Courier New so non-highlighted +samples don't appear out of place. + +CLI output samples are similar to code samples but should be highlighted with +the color they'll output if it is known so that the RFC could also cover +formatting as part of the user experience. + + func example() { + <-make(chan struct{}) + } + + +Note: This template is based on the [RFC template from Hashicorp](https://works.hashicorp.com/articles/rfc-template) used with permission. diff --git a/rfc/1/versions/2024-04-24/diagram.png b/rfc/1/versions/2024-04-24/diagram.png new file mode 100644 index 00000000..bd99cca5 Binary files /dev/null and b/rfc/1/versions/2024-04-24/diagram.png differ diff --git a/rfc/1/versions/2024-04-24/index.md b/rfc/1/versions/2024-04-24/index.md new file mode 100644 index 00000000..22f211f2 --- /dev/null +++ b/rfc/1/versions/2024-04-24/index.md @@ -0,0 +1,551 @@ +# RFC-1: 2024-04-24 + +```{toctree} +:hidden: +:maxdepth: 1 +template +``` + +Definition of the NGFF RFC process + +## Status + +This version of the RFC was sent out for review. + +## Overview + +This RFC defines the high-level decision making process for changes within the +NGFF community. These changes are defined in "Request for Comments" (RFCs) and +therefore this RFC is self-referential: it is following the process that it +itself defines. The overall goal of the process is to make clear how decision +making works with a focus on both speed of development and clarity. It should +be clear after reading the RFC which stakeholder (author, reviewer, editor, +etc.) is responsible for moving decisions along and how much time the +community can expect that decision to take. Not all decisions in the NGFF +community need to follow the RFC process. Clarifications, corrections, and +numerous other changes will proceed following the current GitHub workflow. +However, when decisions reach a certain scale, including significant +specification changes but also changes to the community process itself, RFCs +will provide a mechanism for managing the process. + +## Background + +Growing interest in the NGFF format has also led to increased participation in +the specification process. As such, reaching a consensus for all decisions has +become more difficult. The current approach defined post facto in RFC-0 +follows a full consensus model. Through community meetings and pull requests, +it was expected that all parties agree before a specification change was +considered. This made it especially difficult both for **Authors** as well as +**Reviewers** to know when a suggested change was adopted since PR comments +could re-open a discussion. It also left a significant burden on **Editors** to +draw a line where further discussion was not possible. Such a situation leads +to slower specification evolution and potential deadlocks. + +Led by the need to take on larger challenges and more interested parties, there +has been significant interest within the community to update the process used +in order to bypass these issues. This RFC adapts the well-known [RFC process][1], +which originated in the Internet Engineering Task Force (IETF), for use in the +NGFF community as has been done in a number of other communities ([Rust][2], +[Hashicorp][3], [Tensorflow][4], etc.) More information can be found under: + +- +- + +Ultimately, each RFC is a record of a decision made, either for or against a +proposal, that will be available from the main NGFF webpage. This captures the +currently distributed GitHub conversations into a single, consistent location +that can be reviewed and referenced more widely by the community. + +## Proposal + +The RFC process proposed functions by (1) encouraging submissions from the +community and not blocking their encapsulation as an RFC (2) asking for +descriptive and complete comments from reviewers on these submissions (3) +having clear definitions of what support **must** be received for decisions to +move forward. Goals of this process include (a) maintaining a public record of +the decision making (b) querying acceptance of a proposal throughout the +process and making that readily visible (c) ultimately driving the +implementation of specifications in order to get working solutions into the +hands of the bioimaging community. The process should **NOT** prevent future +discussions on any adopted RFCs but instead will encourage continued +improvement and evolution through discussions of _further_ RFCs. + +## Requirements + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in IETF [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). + +## Stakeholders + +This section lists the major stakeholders in the RFC process and provides an +overview of their responsibilities and involvement in the process. For more +details, see the "Implementation" section below. + +**Authors** propose an idea for the RFC process and socialize the idea, e.g., +through an issue or community call, gaining **Endorsers** They then submit a +pull request to the repository with a document that +they would like to have published as an RFC. This pull request MUST contain a +document under the `rfc/` subdirectory and it SHOULD follow the template +provided. As described under the "DRAFT" section below, this document can be +discussed for clarity following the standard PR process. However, once the +draft has reached a certain stage that it is ready for comments, **Editors** +will merge it as a record of the fact that the suggestion has been made, and +it will then become available on https://ngff.openmicroscopy.org. + +**Endorsers** are non-**Author** supporters of an RFC, listed in a table. +**Reviewers** who have given an "Accept" recommendation are also added to the +table. Rather than a process terminated with a single vote, the RFC process +consists of iterative rounds of gathering **Endorsers**, or sponsors, who +approve the work, building confidence that the community is moving in the +right direction. More information on this strategy can be found under: + . + +**Editors** identify whether a PR should or should not follow the RFC process, +and choosing when a draft is ready to become an RFC. They also choose +appropriate **Reviewers** for an RFC and manage the communication between +**Authors** and **Reviewers**. + +Note: The use of "Editors" in this document is intended as a placeholder. A +future RFC will define the selection and removal of editors. Until that time, +the sole editor is Josh Moore which has been the _de facto_ case since the +inception of NGFF. + +**Reviewers** represent experts in the community whose opinion on an RFC is +necessary in order for the community to feel comfortable that a solution is +advantageous for and implementable by the community. Some number of reviewers +will be asked to provide a review of the RFC along with a recommendation on how +the **Editors** should proceed with the RFC. Depending on the complexity of the +RFC, this process may be iterative and require multiple reviews. + +**Commenters** are other members of the community who are not required to make +a determination on an RFC but are still invited to add their voice to the +decision-making process. + +## Implementation + +This description of the RFC process will refer frequently to the [visual diagram][dia] +Readers may want to familiar themselves with it at this point. +Identifiers such as "D1", "R2", "S3", refer to steps in that diagram. + +### Legend + +Notes regarding specific requirements are called out throughout the text +with the following symbols: + +> * 🕑 The clock symbol specifies definitive wait times within the process. +> * 📂 The folder symbol specifies requirements on additions to the repository, +> for example an implementation or failing test. + +### Phases + +The overall process is broken down into three phases: DRAFT phase before a +proposal becomes an RFC, the RFC phase during which reviews occur, and the SPEC +phase after the RFC has been adopted. + +#### DRAFT + +The DRAFT phase begins when **Authors** propose (D1) a new idea and +subsequently gather support (or "socialize") the idea (D2) before opening a PR +(D3). This phase is meant to promote ideation without premature criticism, though +clarifications can be requested on the GitHub PR (D4) which may require an +update of the PR (D3). Note: the RFC process does not formally make use of GitHub's +"draft" state, though **Editors** MAY choose to ignore such PRs. + +One common clarification is whether or not a given PR should be an RFC. The +RFC process is intended primarily for significant and/or breaking changes to +the specification or the community process itself. Other changes, bug fixes, +etc. are welcome as PRs without a full RFC. If **Authors** are unsure whether +or not a change requires an RFC, they can open an issue or contact the **Editors** +directly. All RFC-targeted PRs SHOULD follow the [current template][tmpl]. + +Comments on how a specification is implemented or what could be proposed +instead from potential **Reviewers** and **Commenters** should come in the +RFC stage, and the **Editor** MAY postpone such discussions when appropriate. + +> 🕑 The iterative clarification phase should be limited to at most a month. + +At the **Editors** discretion (D5), the PR can be merged at which point it +becomes an RFC or closed if there is no interest in progressing with the +proposal. In the latter case, **Authors** can use any feedback to open a new PR +(D3). **Authors** who are unsure if they will be able to shepherd an RFC +throughout the entire process are still invited to open PRs so that they might +be adopted by other members of the community. + +These steps are not significantly different from the previous consensus model +described in RFC-0. However, it is intended that the discussions during this +period are intended to improve the RFC and is not intended to evaluate its +overall value. As described in the next section "RFC", the deeper and more +critical discussions should happen as complete and well-considered reviews +and responses that will help future readers understand the decision-making +process. To this end, **Editors** MAY state on GitHub that a question or +comment is "more appropriate for an official comment". This is not intended to +silence anyone but to manage the overall flow of discussion. + +#### RFC + +Once a PR has been merged to become an RFC, **Editors** are responsible for +adjusting the "Status" in the RFC document to become "RFC phase (R1)", and +identifying and assigning **Reviewers** (R1). **Reviewers** will be given a +period of time for preparing their reviews. If they do not foresee being able +to respond in this time, they SHOULD contact the **Editors** as soon as +possible. + +> 🕑 Reviewers comments should be returned in less than one month. + +**Reviewers** then submit their comments (R2) along with their recommendations +to the **Editors**, either via a public PR adding the review in markdown to the +RFC's subdirectory or by emailing the **Editors** directly. (This latter course +should only be limitedly used when necessary.) + +Possible recommendations from **Reviewers** in ascending order of support are: + +* “Reject” suggests that a **Reviewer** considers there to be no merit to an + RFC. This should be a last recourse. Instead, suggestions in a “Major + changes” recommendation might include attempting an Extension rather than an + RFC so that not all implementations need concern themselves with the matter. +* “Major changes” suggests that a **Reviewer** sees the potential value of an + RFC but will require significant changes before being convinced. Suggestions + SHOULD be provided on how to concretely improve the proposal in order to make + it acceptable and change the **Reviewer**’s recommendation. +* “Minor changes” suggests that if the described changes are made, that + **Editors** can move forward with an RFC without a further review. +* “Accept” is a positive vote and no text review is strictly necessary, though + may be provided to add context to the written record. This is equivalent to + the **Reviewer** joining the list of endorsements. + +Three additional versions of the "Accept" recommendation are available for +**Reviewers** who additionally maintain an implementation of the NGFF +specification to express further support: +* “Plan to implement” with an estimated timeline +* “Implementation begun” with an estimated timeline +* “Implementation complete” with a link to the available code + +Where a review is required, **Reviewers** are free to structure the text in the +most useful way. There is no template but useful sections might include: + +* Summary +* Significant comments and questions +* Minor comments and questions +* Recommendation + +The tone of a review should be cordial and professional. The goal is to communicate +to the **Authors** what it would take to make the RFC acceptable. + +Once **Editors** have received the recommendations and reviews (R3), they +should be added to the repository and the **Authors** should be contacted for a +response. **Authors** are invited at this stage (R4) to update the RFC document +via subsequent PRs. The response SHOULD include a written rebuttal to each of +the reviews. **Editors** include the response in the repository (R5) and +contact **Reviewers** to see if their recommendations have changed. + +> 🕑 Reviewers responses should be returned in less than two weeks. + +This brings a critical, and iterative, decision point (R6). If a "Reject" +recommendation remains, then the RFC is closed. The text remains on the +specification pages for posterity. If sufficient endorsements, +including two in-progress implementations, are available, then the RFC can +progress (S1) to the SPEC phase below. If there are no "Major" objections but +still no consensus, the decision falls to the **Editors** (R7) who may also +move the RFC to the SPEC phase (S0). + +> 📂 Two in-progress implementations required for progressing to S1. + +Otherwise, the RFC iterates through the process again. If the changes made by +the **Authors** are significant, **Reviewers** may be asked to respond again +(R2). Alternatively, **Editors** may send the text back to the **Authors** for +further refinement in order to achieve sufficient endorsement. + +Throughout the RFC phase, non-**Author** supporters of the RFC +are invited to contact the **Editors**, comment on the original PR, and/or open +a new PR to list themselves as **Endorsers** of the RFC. Whether or not the +**Endorser** has begun implementing the RFC is especially of relevance, both +to gauge the degree of confidence but also to allow **Editors** and **Authors** +to contact the **Endorser** regarding any breaking changes. + +#### SPEC + +If an RFC enters the SPEC state via **Editors** approval (S0), an additional +explanation by the **Editors** will be included in the RFC's directory at which +point it is considered equivalent to a **Reviewer** accepted RFC. At this point +(S1), the primary purpose of the RFC becomes driving implementations. Further +clarifications (S2) may be needed. Updates to the RFC as well as the +specification itself will be managed by the **Authors** and the **Editors** in +coordination.**Editors** will also contact remaining implementers (S3) +regarding the status of their implementations and update the endorsements table +accordingly. + +> 🕑 This period may take substantial time to coordinate with implementers, +> but should be limited to approximately a month if possible. + +Once sufficient endorsements, including two released implementations, are +listed, the specification will be considered "adopted". The adopted +specification will be slotted into a release version by the **Editors** and the +**Authors** are encouraged to be involved in that release. + +> 📂 Two released implementations required for being adopted. + +### Decision-making + +Unless otherwise specified in the text, the following considerations are taken +into account when making decisions regarding RFCs: + + * **Prefer working examples**: whether an implementation of an RFC or a + failing test which exposes an issue in a proposal, working examples will + tend to carry more weight in decision making. + * **technical expertise**: all other considerations being equal, feedback + from stakeholders with more technical expertise in a matter under + consideration will tend to carry more weight in decision making. + +## Drawbacks, risks, alternatives, and unknowns + +The primary **drawbacks**, **risks**, and **unknowns** of the proposal revolve +around the cost that the process will place on those participating in it. This +equally points to the primary **alternative** which is to not have such a +process, and "just use GitHub like any other open source project". The +experience of this RFCs author has been that solely relying on issues and PRs +on GitHub does not provide the editorial functions that one would want, such as +deferring and collecting comments, nor do the conversations provide a +consistent whole when revisited after the work on a specification. +Additionally, **Authors** have complained of the burden of managing responses. +So there's a need for *something*, but does this proposal go too far in the +other direction? + +It is certainly true that the formality of the responses asked of the +**Authors** is beyond a typical PR, but the investment for the specifications +as of the writing of this RFC have already been significant, and the RFC +process especially helps **Authors** to structure and coordinate their +proposals. Similarly, the one **Editor** to date (and the author of this RFC) +has also struggled with the lack of an explicit process, and so that too is +likely less of an issue. Is the cost then too high for the **Reviewers**? + +The hope is that by having clear expectations of the times involved in each +step, and following a model which is relatively known (reviewing for a journal) +that **Reviewers** will be better able to judge whether and how frequently they +can be involved. It also puts into place an editorial mechanism to handle the +situation where a **Reviewer** must leave the process early, allowing things to +continue to move forward. + +Will the process be worth the effort? It's unclear. The fact that several other +communities have used variants of the process would imply that it can work. It +might be that we will have to update and further customize the process for the +NGFF community, but by having something explicit as in this RFC-0 it will make +it possible for us to more easily iterate. + +## Abandoned ideas + +As mentioned elsewhere in this document, the current consensus model of +decision making described in RFC-0 is one of the ideas that this RFC would +abandon. An alternative proposal that has been mentioned at various times is +that that someone, likely the Editor, "should just decide". This fiat model, +however, places too much burden on a single individual within the community. + +Looking at the W3C community, there was little information on how each of the +separate working groups (HTML, XML, etc.) conducted their internal business. +Since this proposal intended to begin bottom-up, it was more useful to find a +working internal process like RFC. In the future, looking to the W3C for some +of the higher-level governance may make sense. + +Another potential model described further under "Prior Art" is the "Enhancement +Proposal" model like ZEP. These models work quite similarly to the RFC model. +One major difficulty is that the name "NEP" is already in use by the Napari +community. + +## Prior art and references + +As mentioned in the "Background" section, there are a number of communities +using adaptions of the RFC process which will not be re-listed here. However, +there are also other enhancement processes which are closely related to the +NGFF RFC. Most closely, is the Zarr Enhancement Proposals (ZEP) process within +the Zarr community. Based originally on a combination of the PEP, NEP, and STAC +processes, the ZEP process uses a council of the implementations (ZIC) + +## Future possibilities + +This RFC does not try to define all aspects of the NGFF community process and +instead focuses on the most immediate block which covers what is typically +thought of as the voting process. By establishing this as a foundation, future +RFCs can extend the community process either adding or simplifying structure as +feedback determines. In fact, a few locations in the text are marked as +placeholders and will be updated to reference future RFCs as they are defined. +The following items are therefore considered out of scope for the purposes of +this RFC but future work is intended to evolve the community process. + +### A. Editorial Board + +This proposal does not try to define the set of editors or how they are chosen +and operate. However, with the goal of improving and accelerating the evolution +of the specification, that will clearly be necessary. + +### B. Advisory Board + +Similarly, this proposal does not try to define higher-level governance and +conflict resolution. By default, as a repository within the "OME" GitHub +organization, the community health processes defined in + hold for the NGFF process. However, following +this RFC, additional and specific governance mechanisms can be put in place for +the operation of the NGFF community process. This may or may not follow the +model of other RFC-using communities, e.g., IETF's [IAB][iab] or Fuchsia's +"governing authority". + +### C. Participation requirements + +Membership in the editorial and advisory boards but also as a reviewer +could use a clearer definition of timely participation rules to prevent +anyone individual from blocking the process. + +### D. Multiple tracks + +Currently, there is a single "track" in the [process diagram][dia] through which +all RFCs progress. This can initially be managed by careful choice of the +reviewers. However, possibly as the complexity of review grows, it will become +advantageous to split the set of a reviewers into separate "tracks", such that +specific topic are handling by area experts. The process may also be modified +on a per track basis. For example, an information track which is largely for +training purposes and provides concrete examples to the community could have +an expedited process. + +### E. Versions of specifications and RFCs + +The semantics of versioning for the NGFF specification are not well-defined +enough. Are there development versions ("alpha", "beta")? What should a users +expectations be for such versions? Are some versions more stable than others? +Additionally, are RFCs versioned? Can they be (significantly) updated over +time? + +### F. Required specifications and extensions + +There is currently no clear definition of which parts of the specification, if +any, MUST be implemented by software projects. This has been handled in the +Zarr community by adding a "must understand" flag to extensions to warn +implementations that if they see a particular feature has been included, they +should fail rather than show the user incorrect data. Such a mechanism could +go hand in hand with an as-yet unspecified extension mechanism. This would be +an ideal topic for a following RFC. + +### G. Metadata model + +Like other NGFF specifications, the RFC process has an inherent metadata model +which can be captured and actioned upon within the NGFF repository. A draft +representation of this metadata is presented below: + +``` +{ + "@context": "ngff.openmicroscopy.org/rfc", + "@type": "RFC", + "@id": "####", + // Independent of subsequent RFCs + "title": " + "authors": [ + { + "@type": "http://schema.org/Person", + "@id": ... + } + ], + "status": see enum + "published": TODO: follow some schema.org model? WorkOfArt? + "doi": + "edits": [ + { + "authors": + "date": + "xxx": + } + ], + // May be changed by subsequent RFCs + "obsoletes": [], + "obsoleted-by": [], + "updates": [], + "updated-by":[], +} +``` + +This model is very close to the original IETF RFC model, but omits the +following keywords: + +- Format: we have limited RFCs to Markdown +- Stream: in IETF, different streams are responsible for different parts of + the internet infrastructure. This may be introduced in the future. +- Similarly the STD (“Standard track”), BCP (“best community practice”), FYI + (“informational”) designations are not currently used. + +The possible values for Status are: +- UNKNOWN +- HISTORIC +- INFORMATIONAL +- BEST CURRENT PRACTICE +- EXPERIMENTAL +- PROPOSED STANDARD +- STANDARD + +### H. Misc + +Other possibilities that perhaps do not need an RFC but might be worth +considering: + +- Authors of RFC should be allowed if not encouraged to post their RFCs + to a preprint server. It might then be possible to use the preprint + servers review functionality for the official reviews. +- Other communities like Zarr have discussed having "fast track" specification + changes (e.g., proposing a new codec) that need not follow the entire + process. This may should be considered for certain types of proposals. + +## Examples + +Below is a list of the preceding major decisions within the NGFF community that +have been or will be written up following the RFC template proposed here. + +| RFC | Title | Decision | +|-----|-----------------------------------------|---------------------------------------| +| TBD | Original NGFF including consensus model | Organically developed | +| TBD | Labels | Adopted under consensus model | +| TBD | HCS | Adopted under consensus model | +| TBD | Tables | Withdrawn (i.e., "tabled") | +| TBD | Transforms | In progress under the consensus model | + +Other RFCs that will _likely_ be written can be found under the "Future +possibilities" section. + +## Skipped sections + +This RFC excludes the following optional headers: "Performance", "Backwards +Compatibility", "Security considerations", "Privacy considerations", "Testing", +"UI/UX". + +## Definitions + +Definitions for terms used throughout this RFC have been collected below. + +* Accepted: Specifies that an RFC has passed review and all + implementers should begin implementation if they have not done so already. +* Adopted: An RFC that has been sufficiently implemented to be considered + as active within the community. +* Author: See “Stakeholders” +* Comment: Documents that are included with the RFC discussing the pros and + cons of the proposal in a structured way. Comments from reviewers are + additionally referred to as "reviews". +* Draft: See the related section under “Implementation > Phases” +* Editor: See “Stakeholders” +* Endorsement: +* RFC ("Request for Comment"): A formal proposal following a standardized + template that is made to the NGFF repository. The proposal need not be + accepted to be published online. +* PR: A pull request opened against the ome/ngff repository on GitHub. +* Review: See Comment +* Reviewer: See “Stakeholders” +* ZEP ("Zarr Enhancement Proposal") Decision-making process for the Zarr + specification + +## Changelog + +| Date | Description | Link | +| ---------- | --------------------------------------- | --------------------------------------- | +| 2024-04-24 | Apply changes from comment 1 | | + +[dia]: diagram.png +[tmpl]: template.md +[iab]: https://www.ietf.org/about/groups/iab/ +[1]: https://www.rfc-editor.org/ +[2]: https://github.com/rust-lang/rfcs/blob/master/0000-template.md +[3]: https://works.hashicorp.com/articles/rfc-template +[4]: https://github.com/tensorflow/community/blob/master/rfcs/yyyymmdd-rfc-template.md diff --git a/rfc/1/template.md b/rfc/1/versions/2024-04-24/template.md similarity index 100% rename from rfc/1/template.md rename to rfc/1/versions/2024-04-24/template.md diff --git a/rfc/1/versions/index.md b/rfc/1/versions/index.md new file mode 100644 index 00000000..0ff8a587 --- /dev/null +++ b/rfc/1/versions/index.md @@ -0,0 +1,9 @@ +# Versions + +Key versions of RFC-1 which have been sent for review, etc. + +```{toctree} +:maxdepth: 1 +:glob: +*/index +``` diff --git a/rfc/2/comments/index.md b/rfc/2/comments/index.md new file mode 100644 index 00000000..39447631 --- /dev/null +++ b/rfc/2/comments/index.md @@ -0,0 +1,9 @@ +# Comments + +Additional comments of RFC-2: + +```{toctree} +:maxdepth: 1 +:glob: +*/index +``` diff --git a/rfc/2/index.md b/rfc/2/index.md index 95571c2d..9853587c 100644 --- a/rfc/2/index.md +++ b/rfc/2/index.md @@ -1,26 +1,146 @@ # RFC-2: Zarr v3 +```{toctree} +:hidden: +:maxdepth: 1 +reviews/index +comments/index +responses/index +versions/index +``` + Adopt the version 3 of Zarr for OME-Zarr. ## Status -This RFC is currently in draft state (D3). - -| Role | Name | GitHub Handle | Institution | Date | Status | -| -------- | ------------------- | --------------------------------------------------- | -------------------------------------------------- | ---------- | ----------------------------------------------------------------------- | -| Author | Norman Rzepka | [normanrz](https://github.com/normanrz) | [scalable minds](https://scalableminds.com) | 2024-02-14 | | -| Endorser | Davis Bennett | [d-v-b](https://github.com/d-v-b) | | 2024-02-14 | Endorse | -| Endorser | Kevin Yamauchi | [kevinyamauchi](https://github.com/kevinyamauchi) | ETH Zürich | 2024-02-16 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1947942934) | -| Endorser | John Bogivic | [bogovicj](https://github.com/bogovicj) | HHMI Janelia Research Campus | 2024-02-16 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1948547356) | -| Endorser | Matthew Hartley | [matthewh-ebi](https://github.com/matthewh-ebi) | EMBL-EBI | 2024-02-16 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1948912814) | -| Endorser | Christian Tischer | [tischi](https://github.com/tischi) | EMBL | 2024-02-16 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1949058616) | -| Endorser | Joel Lüthi | [jluethi](https://github.com/jluethi) | BioVisionCenter, University of Zurich | 2024-02-16 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1949333769) | -| Endorser | Constantin Pape | [constantinpape](https://github.com/constantinpape) | University Göttingen | 2024-02-18 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1951318754) | -| Endorser | Will Moore | [will-moore](https://github.com/will-moore) | OME, University of Dundee | 2024-02-19 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1952057704) | -| Endorser | Juan Nunez-Iglesias | [jni](https://github.com/jni) | Biomedicine Discovery Institute, Monash University | 2024-02-20 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1953922897) | -| Endorser | Eric Perlman | [perlman](https://github.com/perlman) | | 2024-02-22 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1960272942) | -| Endorser | Ziwen Liu | [ziw-liu](https://github.com/ziw-liu) | Chan Zuckerberg Biohub | 2024-03-12 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1992588774) | -| Endorser | Lachlan Deakin | [LDeakin](https://github.com/LDeakin) | Australian National University | 2024-03-14 | [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1998594492) | +This RFC is currently in SPEC state (S1). + +```{list-table} Record +:widths: 8, 20, 20, 20, 15, 10 +:header-rows: 1 +:stub-columns: 1 + +* - Role + - Name + - GitHub Handle + - Institution + - Date + - Status +* - Author + - Norman Rzepka + - [normanrz](https://github.com/normanrz) + - [scalable minds](https://scalableminds.com) + - 2024-02-14 + - +* - Endorser + - Davis Bennett + - [d-v-b](https://github.com/d-v-b) + - + - 2024-02-14 + - Endorse +* - Endorser + - Kevin Yamauchi + - [kevinyamauchi](https://github.com/kevinyamauchi) + - ETH Zürich + - 2024-02-16 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1947942934) +* - Endorser + - John Bogivic + - [bogovicj](https://github.com/bogovicj) + - HHMI Janelia Research Campus + - 2024-02-16 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1948547356) +* - Endorser + - Matthew Hartley + - [matthewh-ebi](https://github.com/matthewh-ebi) + - EMBL-EBI + - 2024-02-16 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1948912814) +* - Endorser + - Christian Tischer + - [tischi](https://github.com/tischi) + - EMBL + - 2024-02-16 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1949058616) +* - Endorser + - Joel Lüthi + - [jluethi](https://github.com/jluethi) + - BioVisionCenter, University of Zurich + - 2024-02-16 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1949333769) +* - Endorser + - Constantin Pape + - [constantinpape](https://github.com/constantinpape) + - University Göttingen + - 2024-02-18 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1951318754) +* - Endorser + - Will Moore + - [will-moore](https://github.com/will-moore) + - OME, University of Dundee + - 2024-02-19 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1952057704) +* - Endorser + - Juan Nunez-Iglesias + - [jni](https://github.com/jni) + - Biomedicine Discovery Institute, Monash University + - 2024-02-20 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1953922897) +* - Endorser + - Eric Perlman + - [perlman](https://github.com/perlman) + - + - 2024-02-22 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1960272942) +* - Endorser + - Ziwen Liu + - [ziw-liu](https://github.com/ziw-liu) + - Chan Zuckerberg Biohub + - 2024-03-12 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1992588774) +* - Endorser + - Lachlan Deakin + - [LDeakin](https://github.com/LDeakin) + - Australian National University + - 2024-03-14 + - [Endorse](https://github.com/ome/ngff/pull/227#issuecomment-1998594492) +* - Reviewer + - Melissa Linkert, Sébastien Besson, Chris Allan, Jason Swedlow + - glencoesoftware + - Glencoe Software + - 2024-05-23 + - [Review](./reviews/1/index) +* - Reviewer + - Yaroslav O. Halchenko + - yarikoptic + - Dartmouth College, DANDI Project + - 2024-06-10 + - [Review](./reviews/2/index) +* - Reviewer + - Jeremy Maitin-Shepard + - jbms + - Google + - 2024-04-30 + - [Review](./reviews/3/index) +* - Reviewer + - Melissa Linkert, Sébastien Besson, Chris Allan, Jason Swedlow + - glencoesoftware + - Glencoe Software + - 2024-08-05 + - [Accept](./reviews/1b/index) +* - Reviewer + - Jeremy Maitin-Shepard + - jbms + - Google + - 2024-09-11 + - [Accept](./reviews/3b/index) +* - Reviewer + - Yaroslav O. Halchenko + - yarikoptic + - Dartmouth College, DANDI Project + - 2024-09-11 + - [Accept](./reviews/2b/index) +``` ## Overview @@ -51,17 +171,18 @@ Adopting Zarr v3 in OME-Zarr is a precondition for using sharding. Library support for Zarr v3 is already available for several languages: +- [zarr-python (Python)](https://github.com/zarr-developers/zarr-python) - [tensorstore (C++/Python)](https://github.com/google/tensorstore) -- [zarrita (Python)](https://github.com/scalableminds/zarrita) +- [zarr-java (Java)](https://github.com/zarr-developers/zarr-java) - [zarrita.js (JS)](https://github.com/manzt/zarrita.js) - [zarr3-rs (Rust)](https://github.com/clbarnes/zarr3-rs) -Visualization tools with integrated Zarr implementations are also available: +Visualization tools with integrated Zarr v3 implementations are also available: - [neuroglancer](https://github.com/google/neuroglancer) - [WEBKNOSSOS](https://github.com/scalableminds/webknossos) -Support for other languages is under active development, including C, Java and Python. +Support for other languages is under active development. Libraries will likely prioritize support for v3 over previous versions in the near future. OME-Zarr should therefore adopt the new version for future-proofing. @@ -84,26 +205,7 @@ Implementations can read inner chunks individually. Depending on the choice of codecs and the underlying storage backends, it may be possible to write inner chunks individually. However, in the general case, writing is limited to entire shards. -## Proposal - -This RFC proposes to adopt version 3 of the Zarr format for OME-Zarr. -Images that use the new version of OME-Zarr metadata MUST NOT use Zarr version 2 any more. - -The motivation for making this hard cut is to reduce the burden of complexity for implementations. -Currently, many Zarr library implementations support both versions. -However, in the future they might deprecate support for version 2 or deprioritize it in terms of features and performance. -Additionally, there are OME-Zarr implementations that have their own integrated Zarr stack. -With this hard cut, implementations that only support OME-Zarr versions > 0.5 (TODO: update assigned version number) will not need to implement Zarr version 2 as well. - -From a OME-Zarr user perspective, the hard cut also makes things simpler: ≤ 0.5 => Zarr version 2 and > 0.5 => Zarr version 3 (TODO: update assigned version number). -If users wish to upgrade their data from one OME-Zarr version to another, it would be easy to also migrate the core Zarr metadata to version 3. -This is a fairly cheap operation, because only json files are touched. -Zarr version 2 and 3 metadata could even live side-by-side in the same hierarchy. -There are [scripts available](https://github.com/scalableminds/zarrita/blob/8155761/zarrita/array_v2.py#L452-L559) that can migrate the metadata automatically. - -It is RECOMMENDED that implementations support a range of OME-Zarr versions, including versions based on Zarr version 2. - -### Notable changes in Zarr v3 +### Other notable changes in Zarr v3 There are a few notable changes that Zarr v3 brings for OME-Zarr: @@ -123,15 +225,40 @@ There are a few notable changes that Zarr v3 brings for OME-Zarr: The Zarr specification does not prescribe the support stores for Zarr hierarchies. HTTP(S), File system, S3, GCS, and Zip files are commonly used stores. +## Proposal + +This RFC proposes to adopt version 3 of the Zarr format for OME-Zarr. +Images that use the new version of OME-Zarr metadata MUST NOT use Zarr version 2 any more. + With this proposal all features of the Zarr specification are allowed in OME-Zarr. In the future, the OME-Zarr community MAY decide to restrict the allowed feature set. +The motivation for making this hard cut is to reduce the burden of complexity for implementations. +Currently, many Zarr library implementations support both versions. +However, in the future they might deprecate support for version 2 or deprioritize it in terms of features and performance. +Additionally, there are OME-Zarr implementations that have their own integrated Zarr stack. +With this hard cut, implementations that only support OME-Zarr versions ≥ 0.5 will not need to implement Zarr version 2 as well. + +From an OME-Zarr user perspective, the hard cut also makes things simpler: < 0.5 => Zarr version 2 and ≥ 0.5 => Zarr version 3. +If users wish to upgrade their data from one OME-Zarr version to another, migration tools will be available ([prototype here](https://github.com/scalableminds/zarrita/blob/8155761/zarrita/array_v2.py#L452-L559)). +Migration is a fairly computationally cheap operation, because only json files are touched. + +Due to the existence of large quantities of images in OME-Zarr 0.4, it is RECOMMENDED that implementations continue to support OME-Zarr 0.4 with the underlying Zarr v2. + +OME-Zarr images MUST be consistent in their OME-Zarr and Zarr version. +With this constraint, implementations only need to detect the version of a provided URL or file path once and can assume that all multiscale levels, wells, series images etc. use the same version. + +While technically possible, OME-Zarr 0.5 (with Zarr v3) and OME-Zarr 0.4 (with Zarr v2) metadata could exist side-by-side in a Zarr hierarchy, it is NOT RECOMMENDED. +This may be useful for short periods of time (i.e. during migrations from 0.4 to 0.5), but should not be used longer term. +Multiple metadata versions can lead to conflicts, which may be hard to resolve by implementations. +If implementations encounter 0.4 and 0.5 metadata side-by-side, 0.5 SHOULD be treated preferentially. + ### Changes to the OME-Zarr metadata -While the adoption of Zarr v3 does not strictly require changes to the OME-Zarr metadata, this proposal contains changes to align with community conventions and ease implementation: +While the adoption of Zarr v3 does not strictly require changes to the OME-Zarr metadata, this proposal contains changes to align with [community conventions](https://zarr.dev/zeps/draft/ZEP0004.html#namespacing) and ease implementation: -- OME-Zarr metadata will be stored under a dedicated key in the Zarr array or group attributes. The key will be a well-known URI of the OME-NGFF specification with a version number, e.g. `https://ngff.openmicroscopy.org/0.6`. -- Since the version is already encoded in the new metadata key, the `version` keys in `multiscale`, `plate`, `well` etc. are removed. +- OME-Zarr metadata will be stored under a dedicated `ome` key in the Zarr array or group attributes. +- The version information will be moved from the `multiscale`, `plate`, `well` etc. sections into the new `ome` section. - The `dimension_names` attribute in the Zarr metadata must match the axes names in the OME-Zarr metadata. Finally, this proposal changes the title of the OME-Zarr specification document to "OME-Zarr specification". @@ -142,7 +269,7 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [IETF RFC 2119][IETF RFC 2119] -## Stakeholders (Recommended Header) +## Stakeholders Preliminary work of this RFC has been discussed in: @@ -153,35 +280,22 @@ Preliminary work of this RFC has been discussed in: - several Zarr community calls - several recent OME-NGFF community calls. - - ## Implementation -OME-Zarr implementations can rely on existing Zarr libraries to implement the adoption of Zarr v3. +OME-Zarr implementations can rely on existing Zarr libraries to implement the adoption of Zarr v3. +See [Background](#background) for a list of v3-capable Zarr libraries. -TODO: Provide a reference implementation +Support for the OME-Zarr 0.5 metadata is under development in [ome-zarr-py](https://github.com/ome/ome-zarr-py/pull/383/files) and other implementations. ## Drawbacks, risks, alternatives, and unknowns While it is clear that Zarr v3 will become the predominant version of the specification moving forward, current library support for v3 is still under active development. +An alternative to this proposal would be to [add Zarr v3 support to OME-Zarr 0.4](https://github.com/ome/ngff/pull/249) without changes to the OME-Zarr Metadata. +The contents of the `.zattrs` would simply move to the `attributes` within the `zarr.json`. +There would need to be some transparency for users to know what Zarr versions are supported by an implementation. +Additionally, there would be no opportunity to introduce an `ome` namespace in the attributes that is useful for composability. +