Merge remote-tracking branch 'upstream/main' into spec-rfc2

.gitignore

@@ -5,3 +5,4 @@ _build
 _bikeshed
 .tox
 .vscode
+.*plist

0.1/index.bs

@@ -364,7 +364,7 @@ custom attributes of the plate group under the `plate` key.
 </dl>
 
 For example the following JSON object defines a plate with two acquisition and
-6 wells (2 rows and 3 columns), containing up 2 fields of view per acquistion.
+6 wells (2 rows and 3 columns), containing up 2 fields of view per acquisition.
 
 ```json
 "plate": {

0.2/index.bs

@@ -419,7 +419,7 @@ custom attributes of the plate group under the `plate` key.
 </dl>
 
 For example the following JSON object defines a plate with two acquisition and
-6 wells (2 rows and 3 columns), containing up 2 fields of view per acquistion.
+6 wells (2 rows and 3 columns), containing up 2 fields of view per acquisition.
 
 ```json
 "plate": {

0.3/index.bs

@@ -432,7 +432,7 @@ custom attributes of the plate group under the `plate` key.
 </dl>
 
 For example the following JSON object defines a plate with two acquisition and
-6 wells (2 rows and 3 columns), containing up 2 fields of view per acquistion.
+6 wells (2 rows and 3 columns), containing up 2 fields of view per acquisition.
 
 ```json
 "plate": {

_static/main.js

@@ -0,0 +1,3 @@
+$(document).ready( function () {
+    $('table.datatable').DataTable();
+} );

conf.py

@@ -15,6 +15,8 @@
 
 extensions = ["myst_parser"]
 source_suffix = [".rst", ".md"]
+myst_heading_anchors = 5
+myst_enable_extensions = ["deflist"]
 
 templates_path = ['_templates']
 exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', '.git', '.pytest_cache', '**/.pytest_cache', '**/.tox', 'README.md', 'LICENSE.md', 'CONTRIBUTING.md']

data/index.md

@@ -7,6 +7,7 @@ Data Resources
 | [Cell Painting Gallery](https://github.com/broadinstitute/cellpainting-gallery) | AWS Open Data Program                         | 136          | 20 TB    |
 | [CZB-Zebrahub](https://zebrahub.ds.czbiohub.org/imaging)                 | czbiohub                                             | 5            | 1.2 TB   |
 | [DANDI](https://dandiarchive.org/dandiset/000108) ([identifiers.org][dandi2],[github][dandi3]) | AWS Open Data Program          | 3914         | 355 TB   |
+| [JAX](https://images.jax.org/webclient/userdata/?experimenter=-1) | The Jackson Laboratory          |   17000       | 192 TB   |
 | [Glencoe](https://glencoesoftware.com/ngff)                              | Glencoe Software, Inc.                               | 8            | 165 GB   |
 | [IDR Samples](https://idr.github.io/ome-ngff-samples/)                   | EBI                                                  | 88           | 3 TB     |
 | [MoBIE](https://mobie.github.io/specs/ngff.html)                         | EMBL-HD                                              | 21           | 2 TB     |

index.rst

@@ -24,6 +24,3 @@ Sample NGFF datasets provided by the community can be found under :doc:`data/ind
    specifications/index
    rfc/index
    tools/index
-
-.. raw:: html
-

requirements.txt

@@ -1,3 +1,3 @@
-bikeshed
+bikeshed<4.2
 myst-parser
 sphinx-book-theme

rfc/0/index.md

@@ -1,4 +1,166 @@
-RFC-0
-=====
+# RFC-0: Consensus model
 
-Example
+Original NGFF consensus process
+
+## Status
+
+This is a historical RFC, drafted after the fact, and has
+been outdated by [RFC-1][1].
+
+```{list-table} Record
+:widths: 20, 20, 20, 15, 10
+:header-rows: 1
+:stub-columns: 1
+
+*   - Name
+    - GitHub Handle
+    - Institution
+    - Date
+    - Status
+*   - Josh Moore
+    - [joshmoore](https://github.com/joshmoore)
+    - [German BioImaging, e.V.](https://ror.org/05tpnw772)
+    - 2022-07-11 ([issue-132](https://github.com/ome/ngff/issues/132)) / 2024-08-30 (RFC-0)
+    - Author
+```
+
+## Overview
+
+Versions of the NGFF specification up to and including v0.4 followed a
+consensus model of decision making. This process is being recorded as RFC-0
+to simplify referencing the former process in [RFC-1][2] and elsewhere.
+
+## Background
+
+NGFF development began as a relatively focused group of individuals writing
+specifications. At the time, this group was called "editors". This document
+describes the process that was used for reviewing and finding a consensus on
+their work before [RFC-1][3] was adopted. This period started with the
+first commit 2020-11-17 and covers versions 0.1 through 0.4, while
+version 0.5 was largely developed concurrently with [RFC-1][4].
+
+## Proposal
+
+A general outline of how versions were decided:
+
+* **Proposals** generally gathered initial support during **community  calls** which often led to the drafting of an issue. In the case of v0.1,
+  the first issue was on the zarr-specs repository,
+  [https://github.com/zarr-developers/zarr-specs/issues/50][5].
+* To the extent possible, **implementations** were started as soon as there was
+  a general consensus since it is much easier to show what is intended when
+  there is draft data which requires a writer implementation. This data was
+  often uploaded to a temporary S3 location.
+* Once the **PR was opened**, there was a general call for **feedback**
+  typically via the [https://image.sc][6] **@ngff group** with various iterations.
+  Sometimes a second **community call** occurred at this point.
+* Once no further changes were requested, a final call with a deadline for
+  release was made either via the image.sc group or a community call or both.
+* The tag was pushed and then an attempt to make sure all implementations were
+  aligned followed.
+
+An exception to the above were a small number of "transitional" specifications.
+This includes the `omero` and `bioformats2raw.layout` sections. They differ
+from other proposals in that:
+
+ - They captured a format that was already in use in the wild.
+ - They affected one or more _existing_ versions.
+ - Support in all clients seemed to be less essential, though there was no precise definition of “essential” .
+
+## Requirements
+
+The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
+"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
+interpreted as described in IETF RFC 2119.
+
+## Stakeholders
+
+* **editors**: those authored specifications. Note: this terminology
+  differs from that in [RFC-1][7] and the website and specifications have been updated to reflect this change.
+* **Implementers**: those who maintain a library or client which implements specs (see below)
+* **developer community**: those who build tools that are expected to support NGFFs (e.g., napari, BDV, Viv)
+* **[the ngff group][8]**: those who have actively signed up for calls, feedback, etc.
+* and, of course, the wider community of users.
+
+## Implementation
+
+Typically a specification proposal included a single PR which updated the `latest` version
+of the specification in the `index.bs` file and additionally included examples and schema
+files representing the changes.
+
+Once data could be generated by one of the implementations, there was then a
+general attempt to update all implementing libraries (ome-zarr-py, n5-zarr,
+[validator][9], etc.) and implementing
+clients (MoBIE, ViZarr, napari-ome-zarr, neuroglancer, ITK, etc.). Minimally,
+one update in each of the three primary languages, Java, Python and Javascript,
+was undertaken.
+
+
+### Phases
+
+The phases below were sometimes concurrent and/or in a different order.
+
+* **Proposal**: an informal statement that someone would like to lead an effort
+  to change the specification.
+* **Issue opened** (optional): a slightly more formal statement of the purposes of a
+  Proposal that tracked all of the associated work.
+* **Community call**: an open call, usually on Zoom, announced under
+  [https://forum.image.sc/tag/ome-ngff][10] where a proposal might be discussed.
+* **Hackathon**: an open, often in-person, meeting to work on a Proposal.
+* **Drafting**: period during which the specification is being actively
+  modified.
+* **PR Opened**: a change to the specification including the normative text,
+  examples, schemas and schema tests.
+* **Discussions**: once a formal change was opened, period of time during
+  which comments and feedback were collected and changes made.
+* **PR Merged**: no further changes intended and the specification is
+  considered acceptable to be part of "latest".
+* **Acceptance**: the changes to "latest" are considered acceptable for a
+  release and are ported to the appropriate version directory.
+* **Implementation**: support added to each of the libraries and clients
+  listed above. A differentiation MAY be made between core (MUST) and
+  additional (SHOULD) components.
+* **Release**: an NGFF specification version was formally released by adding the
+  appropriate tag to the NGFF repository.
+
+## Drawbacks, risks, alternatives, and unknowns
+
+Beyond a certain scale, the consensus model does not help maintainers of the
+specification to move discussions forward. There is no clear arbiter of
+decisions, and the timeline of discussions are not specified. A decision may
+have been made, but without a clear record pull requests open for
+implementation are effectively still in play. Additionally, authors were often
+left with many separate questions on GitHub pull requests that needed answering
+with no clear mechanism for managing those.
+
+## Future possibilities
+
+Please see [RFC-1][11].
+
+## Examples
+
+The following PRs updated the  specification under the RFC-0 model:
+* [3][12] labels (v0.1.3)
+* [5][13] high-content-screening (v0.1.4)
+* [40][14] nested storage (0.2.0)
+* [46][15] and [57][16] axes (0.3.0)
+* [64][17] table (closed)
+* [112][18] transitional bioformats2raw (0.4.1)
+
+[1]:	../1/index
+[2]:	../1/index
+[3]:	../1/index
+[4]:	../1/index
+[5]:	https://github.com/zarr-developers/zarr-specs/issues/50
+[6]:	https://image.sc
+[7]:	../1/index
+[8]:	https://forum.image.sc/g/ngff
+[9]:	https://github.com/ome/ome-ngff-validator/
+[10]:	https://forum.image.sc/tag/ome-ngff
+[11]:	../1/index
+[12]:	https://github.com/ome/ngff/pull/3
+[13]:	https://github.com/ome/ngff/pull/5
+[14]:	https://github.com/ome/ngff/pull/40
+[15]:	https://github.com/ome/ngff/pull/46
+[16]:	https://github.com/ome/ngff/pull/57
+[17]:	https://github.com/ome/ngff/pull/64
+[18]:	https://github.com/ome/ngff/pull/112

rfc/1/comment_1.md → rfc/1/comments/1/index.md

@@ -1,4 +1,4 @@
-# Comment to [RFC 1](../1) from @LucaMarconato and @melonora.*
+# RFC-1: Comment 1
 
 | Name                   | GitHub Handle | Institution          |
 |------------------------|---------------|----------------------|

rfc/1/comment_2.md → rfc/1/comments/2/index.md

@@ -1,4 +1,4 @@
-# Comment to [RFC 1](../1) from @thewtex
+# RFC-1: Comment 2
 
 | Name                   | GitHub Handle | Institution          |
 |------------------------|---------------|----------------------|

rfc/1/comments/index.md

@@ -0,0 +1,9 @@
+# Comments
+
+Additional comments of RFC-1:
+
+```{toctree}
+:maxdepth: 1
+:glob:
+*/index
+```