Skip to content

Commit

Permalink
Add STAC catalog
Browse files Browse the repository at this point in the history
## Overview

- Adds STAC to the stack (optional) when ``./components/stac`` is added to ``EXTRA_CONF_DIRS``.

## Changes

  * Service ``stac`` (API) gets added with endpoints ``/twitcher/ows/proxy/stac`` and ``/stac``.

  * STAC catalog can be explored via the ``stac-browser`` component, available under ``/stac-browser``.

  * Image ``crim-ca/stac-app`` is a STAC implementation based on ``stac-utils/stac-fastapi``.

  * Image ``crim-ca/stac-browser`` is a fork of ``radiantearth/stac-browser``.

  * Adds `Magpie` permissions and service for `stac` endpoints.

  * Uses [stac-populator](https://github.com/crim-ca/stac-populator) to populate STAC catalog with sample collection 
  items via [CEDA STAC Generator](https://github.com/cedadev/stac-generator), employed in sample 
  [CMIP Dataset Ingestion Workflows](https://github.com/cedadev/stac-generator-example/tree/master/conf).

## Demo Instance

STAC API : https://stac-dev.crim.ca/stac/
STAC Browser : https://stac-dev.crim.ca/stac-browser/

Note that by default STAC API will return 10 items to reduce payload size. It is however possible to change this limitation by adding `?limit=200` to the URL in order to query 200 items. In the response payload you'll have a link referring to the `next` items, adding a token to the query params in order for STAC API to return next results.

**Sample STAC API collection query using a CLI**

Remove the `-c` flag for global query across any collection.

```
pip install pystac-client
stac-client search https://stac-dev.crim.ca/stac -c c604ffb6d610adbb9a6b4787db7b8fd7 --query "variable_id=txgt_32" "scenario=ssp585"
```

**Sample STAC API global query using CQL via cURL call**

Note that the operators are describe here : https://portal.ogc.org/files/96288

```
curl --location --globoff 'https://stac-dev.crim.ca/stac/search' \
--header 'Content-Type: application/json' \
--data '{
   "filter":{
      "and":[
         {
            "eq":[
               {
                  "property":"freq"
               },
               "MS"
            ]
         },
         {
            "like":[
               {
                  "property":"variable_id"
               },
               "tr_%"
            ]
         }
      ],
      "intersects":[
         {
            "property":"geometry"
         },
         {
            "type":"Polygon",
            "coordinates":[
               [
                  [
                     -140.99778,
                     41.6751050889
                  ],
                  [
                     -140.99778,
                     83.23324
                  ],
                  [
                     -52.6480987209,
                     83.23324
                  ],
                  [
                     -52.6480987209,
                     41.6751050889
                  ],
                  [
                     -140.99778,
                     41.6751050889
                  ]
               ]
            ]
         }
      ],
      "anyinteracts":[
         {
            "property":"datetime"
         },
         [
            "2010-05-03T13:21:30.040Z",
            "2022-05-03T13:21:30.040Z"
         ]
      ]
   }
}'
```

**Get the queryables of the CMIP6 collection, statically created at collection creation**

https://stac-dev.crim.ca/stac/collections/c604ffb6d610adbb9a6b4787db7b8fd7

**Get the queryables of the CMIP6 collection, dynamically created at query time**

https://stac-dev.crim.ca/stac/collections/c604ffb6d610adbb9a6b4787db7b8fd7/queryables

**Get the queryables of the union of the CMIP5 and CMIP6 collections, dynamically created at query time**

https://stac-dev.crim.ca/stac/queryables?collections=0798aa197d54eb4332767a5a4077fb0f,c604ffb6d610adbb9a6b4787db7b8fd7

daccs_configs_branch: stac_populator
daccs_skip_ci: true

fyi @huard @mishaschwartz
  • Loading branch information
mishaschwartz authored Aug 10, 2023
2 parents bc3273c + 5cb3afb commit baec590
Show file tree
Hide file tree
Showing 7 changed files with 20 additions and 15 deletions.
6 changes: 3 additions & 3 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.27.1
current_version = 1.28.0
commit = True
tag = False
tag_name = {new_version}
Expand Down Expand Up @@ -30,11 +30,11 @@ search = {current_version}
replace = {new_version}

[bumpversion:file:RELEASE.txt]
search = {current_version} 2023-07-10T19:20:25Z
search = {current_version} 2023-08-10T19:23:14Z
replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ}

[bumpversion:part:releaseTime]
values = 2023-07-10T19:20:25Z
values = 2023-08-10T19:23:14Z

[bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template]
search = 'version': '{current_version}'
Expand Down
5 changes: 5 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@
[Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest)
------------------------------------------------------------------------------------------------------------------

[//]: # (list changes here, using '-' for each new entry, remove this when items are added)

[1.28.0](https://github.com/bird-house/birdhouse-deploy/tree/1.28.0) (2023-08-10)
------------------------------------------------------------------------------------------------------------------

## Changes
- Adds [STAC](https://github.com/crim-ca/stac-app) to the stack (optional) when ``./components/stac``
is added to ``EXTRA_CONF_DIRS``. For more details, refer to
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Generic variables
override SHELL := bash
override APP_NAME := birdhouse-deploy
override APP_VERSION := 1.27.1
override APP_VERSION := 1.28.0

# utility to remove comments after value of an option variable
override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")
Expand Down
8 changes: 4 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ for a full-fledged production platform.
* - releases
- | |latest-version| |commits-since|

.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.27.1.svg
.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.28.0.svg
:alt: Commits since latest release
:target: https://github.com/bird-house/birdhouse-deploy/compare/1.27.1...master
:target: https://github.com/bird-house/birdhouse-deploy/compare/1.28.0...master

.. |latest-version| image:: https://img.shields.io/badge/tag-1.27.1-blue.svg?style=flat
.. |latest-version| image:: https://img.shields.io/badge/tag-1.28.0-blue.svg?style=flat
:alt: Latest Tag
:target: https://github.com/bird-house/birdhouse-deploy/tree/1.27.1
:target: https://github.com/bird-house/birdhouse-deploy/tree/1.28.0

.. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
:alt: ReadTheDocs Build Status (latest version)
Expand Down
2 changes: 1 addition & 1 deletion RELEASE.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.27.1 2023-07-10T19:20:25Z
1.28.0 2023-08-10T19:23:14Z
8 changes: 4 additions & 4 deletions birdhouse/config/canarie-api/docker_configuration.py.template
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,8 @@ SERVICES = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '1.27.1',
'releaseTime': '2023-07-10T19:20:25Z',
'version': '1.28.0',
'releaseTime': '2023-08-10T19:23:14Z',
'institution': 'Ouranos',
'researchSubject': 'Climatology',
'supportEmail': '${SUPPORT_EMAIL}',
Expand Down Expand Up @@ -142,8 +142,8 @@ PLATFORMS = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '1.27.1',
'releaseTime': '2023-07-10T19:20:25Z',
'version': '1.28.0',
'releaseTime': '2023-08-10T19:23:14Z',
'institution': 'Ouranos',
'researchSubject': 'Climatology',
'supportEmail': '${SUPPORT_EMAIL}',
Expand Down
4 changes: 2 additions & 2 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,9 @@
# built documents.
#
# The short X.Y version.
version = '1.27.1'
version = '1.28.0'
# The full version, including alpha/beta/rc tags.
release = '1.27.1'
release = '1.28.0'

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down

0 comments on commit baec590

Please sign in to comment.