Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collection enhancements #53

Merged
merged 12 commits into from
Feb 28, 2024
4 changes: 4 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
## [Unreleased](https://github.com/crim-ca/stac-populator) (latest)

<!-- insert list items of new changes here -->
- Adding ability to add collection level assets
- Adding ability to add collection level links
- Adding collection links to `CMIP6_UofT`
- Adding an end date to `CMIP6_UofT`'s temporal extent for better rendering in STAC Browser
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you show how 2500-01-01 makes it better for rendering?
The current one seems fine to me:

https://redoak.cs.toronto.edu/stac-browser/collections/CMIP6_UofT
image

If the final date of CMIP6 is unknown, it looks like the "until present" is ideal.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "until present" is not ideal because it's not true. The final date for CMIP6 data is known, the maximum a simulation in the archive goes up to is year 2500. When I originally had "null" I was expecting something like STAC browser to show an undetermined/open end date, not something wrong like "until present".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a specific date imposed by CMIP6, then it is fine. I'm not familiar with that granular level of details about it.


## [0.6.0](https://github.com/crim-ca/stac-populator/tree/0.6.0) (2024-02-22)

Expand Down
15 changes: 14 additions & 1 deletion STACpopulator/implementations/CMIP6_UofT/collection_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,17 @@ description: Coupled Model Intercomparison Project phase 6
keywords: ['CMIP', 'CMIP6', 'WCRP', 'Climate Change']
license: "CC-BY-4.0"
spatialextent: [-180, -90, 180, 90]
temporalextent: ['1850-01-01', null]
temporalextent: ['1850-01-01', '2500-01-01']

links:
homepage:
rel: about
title : Project homepage
target : https://wcrp-cmip.org/cmip-phase-6-cmip6/
media_type: text/html
license:
rel: license
title : License
target : https://raw.githubusercontent.com/WCRP-CMIP/CMIP6_CVs/main/LICENSE-CC-BY
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use a dict instead of a list if the link name is dropped anyway?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea. Done.

media_type: text/plain

35 changes: 34 additions & 1 deletion STACpopulator/populator_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import os
from abc import ABC, abstractmethod
from datetime import datetime
from typing import Any, MutableMapping, Optional, Type, Union
from typing import Any, Dict, List, MutableMapping, Optional, Type, Union

import pystac
from requests.sessions import Session
Expand Down Expand Up @@ -135,13 +135,46 @@ def create_stac_collection(self) -> dict[str, Any]:
)
self._collection_info["extent"] = pystac.Extent(sp_extent, tmp_extent)
self._collection_info["summaries"] = pystac.Summaries({"needs_summaries_update": ["true"]})

# Add any assets if provided in the config
self._collection_info["assets"] = self.__make_collection_assets()

# Construct links if provided in the config. This needs to be done before constructing a collection object.
collection_links = self.__make_collection_links()

collection = pystac.Collection(**self._collection_info)

if collection_links:
collection.add_links(collection_links)
collection.add_links(self._ingest_pipeline.links)
collection_data = collection.to_dict()
self.publish_stac_collection(collection_data)
return collection_data

def __make_collection_links(self) -> List[pystac.Link]:
"""Create collection level links based on data read in from the configuration file.

:return: List of pystac Link objects
:rtype: List[pystac.Link]
"""
links = []
config_links = self._collection_info.pop("links", {})
for link_name, link_info in config_links.items():
links.append(pystac.Link(**link_info))
return links

def __make_collection_assets(self) -> Dict[str, pystac.Asset]:
"""Creates collection level assets based on data read in from the configuration file.

:return: Dictionary of pystac Asset objects
:rtype: Dict[pystac.Asset]
"""
pystac_assets = {}
if "assets" in self._collection_info:
for asset_name, asset_info in self._collection_info["assets"].items():
pystac_assets[asset_name] = pystac.Asset(**asset_info)
return pystac_assets

def publish_stac_collection(self, collection_data: dict[str, Any]) -> None:
post_stac_collection(self.stac_host, collection_data, self.update, session=self._session)

Expand Down
Loading