Skip to content

Commit

Permalink
Merge branch 'main' into doc/v3-technical-design
Browse files Browse the repository at this point in the history
  • Loading branch information
jhamman committed Jan 16, 2024
2 parents ab3f781 + 1d56da0 commit 8ec98c3
Show file tree
Hide file tree
Showing 10 changed files with 70 additions and 15 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ body:
attributes:
label: Python Version
description: Version of Python interpreter
placeholder: 3.8.5, 3.9, 3.10, etc.
placeholder: 3.9, 3.10, 3.11, etc.
validations:
required: true
- type: input
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11']
numpy_version: ['>=1.22.0', '==1.20.*']
python-version: ['3.9', '3.10', '3.11']
numpy_version: ['>=1.22.0', '==1.21.*']
exclude:
- python-version: '3.10'
numpy_version: '==1.20.*'
numpy_version: '==1.21.*'
- python-version: '3.11'
numpy_version: '==1.20.*'
numpy_version: '==1.21.*'
services:
redis:
image: redis
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/releases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- uses: actions/[email protected]
name: Install Python
with:
python-version: '3.8'
python-version: '3.9'

- name: Install PyBuild
run: |
Expand All @@ -36,7 +36,7 @@ jobs:
else
echo "All seem good"
fi
- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: releases
path: dist
Expand All @@ -45,7 +45,7 @@ jobs:
needs: [build_artifacts]
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@v3
- uses: actions/download-artifact@v4
with:
name: releases
path: dist
Expand All @@ -60,7 +60,7 @@ jobs:
runs-on: ubuntu-latest
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
steps:
- uses: actions/download-artifact@v3
- uses: actions/download-artifact@v4
with:
name: releases
path: dist
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/windows-testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
strategy:
fail-fast: True
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11']
python-version: ['3.9', '3.10', '3.11']
steps:
- uses: actions/checkout@v4
with:
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ default_language_version:
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: 'v0.1.9'
rev: 'v0.1.13'
hooks:
- id: ruff
- repo: https://github.com/psf/black
Expand Down
26 changes: 26 additions & 0 deletions TEAM.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## Active core-developers
- @joshmoore (Josh Moore)
- @jni (Juan Nunez-Iglesias)
- @rabernat (Ryan Abernathey)
- @jhamman (Joe Hamman)
- @d-v-b (Davis Bennett)
- @jakirkham (jakirkham)
- @martindurant (Martin Durant)
- @normanrz (Norman Rzepka)

## Emeritus core-developers
- @alimanfoo (Alistair Miles)
- @shoyer (Stephan Hoyer)
- @ryan-williams (Ryan Williams)
- @jrbourbeau (James Bourbeau)
- @mzjp2 (Zain Patel)
- @grlee77 (Gregory Lee)

## Former core-developers
- @jeromekelleher (Jerome Kelleher)
- @tjcrone (Tim Crone)
- @funkey (Jan Funke)
- @shikharsg
- @Carreau (Matthias Bussonnier)
- @dazzag24
- @WardF (Ward Fisher)
5 changes: 5 additions & 0 deletions docs/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,15 @@ Docs
* Minor tweak to advanced indexing tutorial examples.
By :user:`Ross Barnowski <rossbar>` :issue:`1550`.

* Added section about accessing zip files that are on s3.
By :user:`Jeff Peck <jeffpeck10x>` :issue:`1613`.

Maintenance
~~~~~~~~~~~

* Drop Python 3.8 and NumPy 1.20
By :user:`Josh Moore <joshmoore>`; :issue:`1557`.

* Cache result of ``FSStore._fsspec_installed()``.
By :user:`Janick Martinez Esturo <ph03>` :issue:`1581`.

Expand Down
25 changes: 25 additions & 0 deletions docs/tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1000,6 +1000,31 @@ separately from Zarr.

.. _tutorial_copy:

Accessing Zip Files on S3
~~~~~~~~~~~~~~~~~~~~~~~~~

The built-in `ZipStore` will only work with paths on the local file-system, however
it is also possible to access ``.zarr.zip`` data on the cloud. Here is an example of
accessing a zipped Zarr file on s3:

>>> s3_path = "s3://path/to/my.zarr.zip"
>>>
>>> s3 = s3fs.S3FileSystem()
>>> f = s3.open(s3_path)
>>> fs = ZipFileSystem(f, mode="r")
>>> store = FSMap("", fs, check=False)
>>>
>>> # cache is optional, but may be a good idea depending on the situation
>>> cache = zarr.storage.LRUStoreCache(store, max_size=2**28)
>>> z = zarr.group(store=cache)

This store can also be generated with ``fsspec``'s handler chaining, like so:

>>> store = zarr.storage.FSStore(url=f"zip::{s3_path}", mode="r")

This can be especially useful if you have a very large ``.zarr.zip`` file on s3
and only need to access a small portion of it.

Consolidating metadata
~~~~~~~~~~~~~~~~~~~~~~

Expand Down
2 changes: 1 addition & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ channels:
dependencies:
- wheel
- numcodecs >= 0.6.4
- numpy >= 1.20
- numpy >= 1.21
- pip
- pip:
- asciitree
Expand Down
5 changes: 2 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ readme = { file = "README.md", content-type = "text/markdown" }
maintainers = [
{ name = "Alistair Miles", email = "[email protected]" }
]
requires-python = ">=3.8"
requires-python = ">=3.9"
dependencies = [
'asciitree',
'numpy>=1.20,!=1.21.0',
'numpy>=1.21.1',
'fasteners',
'numcodecs>=0.10.0',
]
Expand All @@ -30,7 +30,6 @@ classifiers = [
'Topic :: Software Development :: Libraries :: Python Modules',
'Operating System :: Unix',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
Expand Down

0 comments on commit 8ec98c3

Please sign in to comment.