Code

ZarrWriter speedup#195

Code

Merged

i-jey merged 25 commits intomasterczbiohub-sf/ulc-malaria-scope:masterfrom zarrwriter-xtra-bigczbiohub-sf/ulc-malaria-scope:zarrwriter-xtra-bigCopy head branch name to clipboardDec 23, 2022

+90-80

Commits

Commits on Dec 15, 2022

• use one array for all images

  Axel-Jacobsen

  committed
  99fae7b

  Copy full SHA for 99fae7b

• update docs

  Axel-Jacobsen

  committed
  938384c

  Copy full SHA for 938384c

• use constants for zarr array dimensions

  Axel-Jacobsen

  committed
  213a249

  Copy full SHA for 213a249

• check for exceptions when closing zarrwriter

  Axel-Jacobsen

  committed
  c2ec078

  Copy full SHA for c2ec078

• TIL list appends are atomic

  Axel-Jacobsen

  committed
  81fb8de

  Copy full SHA for 81fb8de

• autoformat

  Axel-Jacobsen

  committed
  36dd141

  Copy full SHA for 36dd141

• logger.error instead of logger

  Axel-Jacobsen

  committed
  c40899f

  Copy full SHA for c40899f

• w is overwrite

  Axel-Jacobsen

  committed
  92b55b1

  Copy full SHA for 92b55b1

• clean up a little bit

  Axel-Jacobsen

  committed
  8f68471

  Copy full SHA for 8f68471

• Merge branch 'zarrwriter-xtra-big' of https://github.com/czbiohub/ulc-malaria-scope into zarrwriter-xtra-big

  Axel-Jacobsen

  committed
  2771a58

  Copy full SHA for 2771a58

Commits on Dec 16, 2022

• sketch out a chunkwriter class

  Axel-Jacobsen

  committed
  1842ad9

  Copy full SHA for 1842ad9

• remove ChunkWriter from zarrwriter

  Axel-Jacobsen

  committed
  166712a

  Copy full SHA for 166712a

Commits on Dec 17, 2022

• Merge branch 'master' into zarrwriter-xtra-big

  Axel-Jacobsen

  authored
  d53e60f

  Copy full SHA for d53e60f

• autoformat

  Axel-Jacobsen

  committed
  c23e4ce

  Copy full SHA for c23e4ce

• bugfixes

  Axel-Jacobsen

  committed
  a14d71e

  Copy full SHA for a14d71e

• bugfixes 2

  Axel-Jacobsen

  committed
  888acc6

  Copy full SHA for 888acc6

• use image counter in dev_run.py

  

  i-jey

  authored and

  i-jey

  committed
  1dc22a3

  Copy full SHA for 1dc22a3

Commits on Dec 19, 2022

• Merge branch 'master' into zarrwriter-xtra-big

  

  Michelle Khoo

  committed
  b3148ac

  Copy full SHA for b3148ac

• maxowkresr = 1

  Axel-Jacobsen

  committed
  09bf6de

  Copy full SHA for 09bf6de

Commits on Dec 20, 2022

• if using a .zip filename, zarr needs it to be closed explicitly, otherwise we get a corrupt zipfile error

  

  i-jey

  authored and

  i-jey

  committed
  e9ab4dc

  Copy full SHA for e9ab4dc

Commits on Dec 21, 2022

• don't use zipstore, use .zarr extension instead

  

  i-jey

  authored and

  i-jey

  committed
  9bd0125

  Copy full SHA for 9bd0125

• Merge branch 'master' into zarrwriter-xtra-big

  Axel-Jacobsen

  committed
  c34d858

  Copy full SHA for c34d858

Commits on Dec 22, 2022

• Merge branch 'master' into zarrwriter-xtra-big

  Axel-Jacobsen

  committed
  1f50aab

  Copy full SHA for 1f50aab

Commits on Dec 23, 2022

• switch back to zipstore

  

  i-jey

  authored and

  i-jey

  committed
  eb2e20d

  Copy full SHA for eb2e20d

• add back in data storage throttle for dev_run - even with these changes, we can't keep up at 53fps

  

  i-jey

  authored and

  i-jey

  committed
  7a9863e

  Copy full SHA for 7a9863e