✨ PySTACAPIItemLister to list STAC Items matching STAC API search

[weiji14]

An iterable-style DataPipe to list STAC Items matching a STAC API search query! Calls pystac_client.ItemSearch.items() to yield pystac.Item instances.

Preview at https://zen3geo--111.org.readthedocs.build/en/111/api.html#zen3geo.datapipes.PySTACAPIItemLister

Usage:

import pystac_client

from torchdata.datapipes.iter import IterableWrapper
from zen3geo.datapipes import PySTACAPIItemLister

# List STAC Items from a STAC API query
query = dict(
    bbox=[57.2, -20.6, 57.9, -19.9],  # xmin, ymin, xmax, ymax
    datetime=["2023-01-01T00:00:00Z", "2023-01-31T00:00:00Z"],
    collections=["s2_l2a"],
)
dp = IterableWrapper(iterable=[query])
dp_pystac_client = dp.search_for_pystac_item(
    catalog_url="https://explorer.digitalearth.africa/stac/"
)
dp_pystac_item_list = dp_pystac_client.list_pystac_items_by_search()

# Loop or iterate over the DataPipe stream
it = iter(dp_pystac_item_list)
stac_item = next(it)

print(stac_item)
# <Item id=ec16dbf6-9729-5a8f-9d72-5e83a8b9f30d>

print(stac_item.properties)
# {'title': 'S2B_MSIL2A_20230103T062449_N0509_R091_T40KED_20230103T075000',
#  'gsd': 10,
#  'proj:epsg': 32740,
#  'platform': 'sentinel-2b',
#  'view:off_nadir': 0,
#  'instruments': ['msi'],
#  'eo:cloud_cover': 0.02,
#  'odc:file_format': 'GeoTIFF',
#  'odc:region_code': '40KED',
#  'constellation': 'sentinel-2',
#  'sentinel:sequence': '0',
#  'sentinel:utm_zone': 40,
#  'sentinel:product_id': 'S2B_MSIL2A_20230103T062449_N0509_R091_T40KED_20230103T075000',
#  'sentinel:grid_square': 'ED',
#  'sentinel:data_coverage': 28.61,
#  'sentinel:latitude_band': 'K',
#  'created': '2023-01-03T06:24:53Z',
#  'sentinel:valid_cloud_cover': True,
#  'sentinel:boa_offset_applied': True,
#  'sentinel:processing_baseline': '05.09',
#  'proj:shape': [10980, 10980],
#  'proj:transform': [10.0, 0.0, 499980.0, 0.0, -10.0, 7900000.0, 0.0, 0.0, 1.0],
#  'datetime': '2023-01-03T06:24:53Z',
#  'cubedash:region_code': '40KED'}

TODO:

• Initial implementation with a doctest and unit test
• Some minor documentation fixes

Notes:

• Why not just use something like:

  def get_all_items(item_search: pystac_client.ItemSearch) -> pystac.ItemCollection:
      return item_search.items()
  
  dp_pystac_item_list = dp_pystac_client.flatmap(fn=get_all_items)

  ? The issue is that FlatMapper doesn't implement the __len__ function (see https://github.com/pytorch/data/blob/v0.6.1/torchdata/datapipes/iter/transform/callable.py#L163-L164), which would break some downstream DataPipes that rely on having a proper __len__.
• Originally wanted a single DataPipe that would produce a list of pystac.Item objects from either ItemSearch or ItemCollection. However, pystac_client.ItemSearch uses .items() (a callable) while pystac.ItemCollection uses .items (not callable), and would necessitate some messy if-then/try-except statements. Hence why the list_pystac_items_by_search functional name, because there might be a list_pystac_items_by_collection for the pystac.ItemCollection.items in the future (if needed).

Part of #48. Extends #59.

[weiji14]

Considered using this try-except statement in case item_search.matched() returns None and raises a TypeError: unsupported operand type(s) for +: 'int' and 'NoneType', but couldn't find a suitable STAC collection at https://radiantearth.github.io/stac-browser/ that doesn't implement .matched(). Leaving this out for now, but may add this back in if someone encounters the TypeError (and files a bug report that can be included in a regression test)!

Suggested change

	return sum(item_search.matched() for item_search in self.source_datapipe)
	try:
	return sum(item_search.matched() for item_search in self.source_datapipe)
	except TypeError: # unsupported operand type(s) for +: 'int' and 'NoneType'
	return sum(len(item_search.items()) for item_search in self.source_datapipe)