Element Metadata Implementation

[Patowhiz]

Overview

I propose we adopt a hierarchical modeling approach for the storage of elements metadata, aligning with the structure outlined by the World Meteorological Organization's Global Climate Observing System (WMO GCOS) Essential Climate Variables (ECVs). This decision is inspired by the comprehensive table available at GCOS Essential Climate Variables. This approach will involve structuring our metadata around four key levels: domains, subdomains, types, and elements.

Proposed Structure

• Domains: The highest categorization level, representing the broadest classification of climate variables; Atmosphere, Land and Ocean.
• Subdomains: Subcategories within domains that offer more specific classification (e.g. Surface, Upper-air etc.).
• Types: Even more specific classification within subdomains, grouping variables by their nature (e.g. Temperature etc.).
• Elements: The most granular level, representing individual climate variables (e.g. Maximum Temperature etc.).

For example, the element "Maximum Temperature" will be categorized as follows:

• Domain: Atmosphere
• Subdomain: Surface
• Type: Temperature
• Element: Maximum Temperature

Rationale

The adoption of the WMO GCOS structure for our metadata organization is expected to significantly streamline and enhance the user experience in terms of quality control and analysis of climate data elements. This structured approach ensures a clear, consistent, and logical hierarchy of climate variables, facilitating easier navigation, understanding, and manipulation of climate data. It also allows us to adopt standard element naming conventions.

Element Setup, Addition and Editing

• We intend to populate the elements table with standard element names during the setup phase. This initial setup ensures consistency and adherence to global standards from the outset.
• Upper limit, lower limit and activated are the only fields that are expected to be editable by users.
• Editing default elements is restricted. That default elements are created by Climsoft as part of its setup.
• Adding new element is allowed. User added elements will start from a specific element id range.
• Adding element type is allowed. This should be carefully evaluated.

Element Naming Conventions

• To further standardize our approach, I suggest our naming to compliment standard element naming conventions as outlined by; The NERC Vocabulary Server implemented by CF Standard Names and WMO code Registry.

These conventions will guide our naming process, ensuring our names and descriptions are interoperable and consistent with international standards.

Element Units

• All observation data will be stored using standard units. Conversion to the element standard unit will occur during ingestion. Data sources differing from the standard units must provide unit conversion metadata to ensure accuracy and consistency across our datasets.

Temporal Resolution Handling

It's important to note that the temporal resolution aspect will not be directly associated with the elements' metadata. Instead, temporal resolution will be associated with the observation itself. This means we will not differentiate elements in the metadata based on their temporal resolution (e.g., "Maximum Temperature Daily" vs. "Maximum Temperature Hourly"). The period aspect of the observation data table will take care of temporal resolution. This approach ensures a streamlined elements table, avoiding unnecessary duplication and simplifying the structure for users.

Expected Benefits:

• Simplistic and Consistent User Functionality: Users will benefit from a straightforward and uniform interface that logically organizes climate variables, making data more accessible and easier to analyze.
• Enhanced Quality Control: The clear definitions of relationships among climate variables, combined with a separate handling of temporal resolution, will aid in the development of more targeted and effective quality control measures.
• Efficient Data Analysis: Analysts will be able to quickly identify and correlate related climate variables across different levels of the hierarchy, improving the efficiency and depth of climate data analysis, while also easily adapting to the specific temporal resolution of observations.

Implementation Considerations

• We will need to carefully map all current desktop and institution's data elements to the new hierarchical structure, ensuring accuracy and consistency.
• The design of the observation data table must accommodate the separation of temporal resolution from the elements' metadata, ensuring flexibility and clarity in data handling.
• User interface and experience design will play a crucial role in making the hierarchical structure and the approach to temporal resolution intuitive and user-friendly.
• Training and documentation will be essential to help users understand and leverage the new structure and temporal resolution handling for quality control and analysis.

Request for Comments

I invite all team members to review this proposed approach, including our strategy for handling temporal resolution, and provide feedback, suggestions, or concerns. Your input is crucial to refining our implementation strategy and ensuring that we fully leverage the potential of the GCOS structure to enhance our climate data management and analysis capabilities.

[Patowhiz]

After discussions with team, we agreed that we should not limit ourselves to the default GCOS Climate Variables classification. We should allow users to extend the classification in ways that are consistent with the structure. We will probably not aim to build in the feature that allows users to add their own subdomains and types in the first version of the web, but in versions that follow we will have such a feature.