diff --git a/README.md b/README.md index dd649cc1..e61fac83 100644 --- a/README.md +++ b/README.md @@ -4,31 +4,39 @@ ![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg) ![CI](https://github.com/alan-turing-institute/clim-recal/actions/workflows/ci.yaml/badge.svg) -Welcome to `clim-recal`, a specialized resource designed to tackle systematic errors or biases in **Regional Climate Models (RCMs)**. As researchers, policy-makers, and various stakeholders explore publicly available RCMs, they need to consider the challenge of biases that can affect the accurate representation of climate change signals. +Welcome to `clim-recal`, a specialised resource which provides a data-processing pipeline for extracting parts of the **UK Climate Projections 2018 Convection Permitting model (UKCP18-CPM)** in order to apply and assess **bias correction methods** via adjustment to and comparison with the **HadUK-Grid**. -`clim-recal` provides both a **broad review** of available bias correction methods as well as **software**, **practical tutorials** and **guidance** that helps users apply these methods methods to various datasets. -`clim-recal` is an **extensive software library and guide to application of Bias Correction (BC) methods**: +`clim-recal:` -- Contains accessible information about the [why and how of bias correction for climate data](#why-bias-correction) -- Is a software library for for the application of BC methods (see our full pipeline for bias-correction of the ground-breaking local-scale (2.2km) [Convection Permitting Model (CPM)](https://www.metoffice.gov.uk/pub/data/weather/uk/ukcp18/science-reports/UKCP-Convection-permitting-model-projections-report.pdf). `clim-recal` brings together different software packages in `python` and `R` that implement a variety of bias correction methods, making it easy to apply them to data and compare their outputs. +- Is a software library for pre-processing climate data to ready it for bias-correction - Was developed in partnership with the MetOffice to ensure the propriety, quality, and usability of our work - Provides a framework for open additions of new software libraries/bias correction methods (in planning) -# Overview: Bias Correction Pipeline +# Overview: Data-processing Pipeline -`clim-recal` is a debiasing pipeline, with the following steps: +Regional climate models (RCMs) contain systematic errors, or biases in their output [^1]. Biases arise in RCMs for a number of reasons, such as the assumptions in the general circulation models (GCMs), and in the downscaling process from GCM to RCM. + +Researchers, policy-makers and other stakeholders wishing to use publicly available RCMs need to consider a range of "bias correction” methods (sometimes referred to as "bias adjustment" or "recalibration"). +Bias correction methods offer a means of adjusting the outputs of RCM in a manner that might better reflect future climate change signals whilst preserving the natural and internal variability of climate [^2]. + +However, in order to apply and assess these methods, the climate model of interest needs to be overlaid to corresponding observation data. This can be a time-consuming and laborious process where data is spatially and temporally very granular. + + +The `clim-recal` pipeline addresses this by providing preprocessed data, including the innovative [UKCP18-CPM datasets](#the-datasets), to facilitate the assessment of these methods on aligned, reprojected data, without requiring the whole (very large) dataset. + +`clim-recal` is a data-processing pipeline, with the following steps: 1. **Set-up & data download** *We provide custom scripts to facilitate download of data* 2. **Preprocessing** *This includes reprojecting, resampling & splitting the data prior to bias correction* -3. **Apply bias correction** - *Our pipeline embeds two distinct methods of bias correction* -4. **Assess the debiased data** - *We have developed a way to assess the quality of the debiasing step across multiple alternative methods* -For a quick start on bias correction, refer to our [pipeline guide](python/README.md). + +For a quick start on installing and running the pipeline, refer to our [pipeline guide](python/README.md). + +Our work is however, just like climate data, intended to be dynamic, and we welcome collaboration from researchers who wish to further our aims! + # Documentation @@ -44,7 +52,6 @@ We are in the process of developing comprehensive documentation for our code bas ## To use `clim-recal` programmatically - There are extensive [`API Reference`](docs/reference) within the python code. -- Comments within `R` scripts ## To contribute to `clim-recal` @@ -52,42 +59,35 @@ We are in the process of developing comprehensive documentation for our code bas # The Datasets -## UKCP18 +## UKCP18-CPM The [UK Climate Projections 2018 (UKCP18)](https://www.metoffice.gov.uk/research/approach/collaboration/ukcp) dataset offers insights into the potential climate changes in the UK. UKCP18 is an advancement of the UKCP09 projections and delivers the latest evaluations of the UK's possible climate alterations in land and marine regions throughout the 21st century. This crucial information aids in future Climate Change Risk Assessments and supports the UK’s adaptation to climate change challenges and opportunities as per the National Adaptation Programme. -## HADS -[HadUK-Grid](https://www.metoffice.gov.uk/research/climate/maps-and-data/data/haduk-grid/haduk-grid) is a comprehensive collection of climate data for the UK, compiled from various land surface observations across the country. This data is organized into a uniform grid to ensure consistent coverage throughout the UK at up to 1km x 1km resolution. The dataset, spanning from 1836 to the present, includes a variety of climate variables such as air temperature, precipitation, sunshine, and wind speed, available on daily, monthly, seasonal, and annual timescales. +We make use of the [Convection Permitting Model (CPM)](https://www.metoffice.gov.uk/pub/data/weather/uk/ukcp18/science-reports/UKCP-Convection-permitting-model-projections-report.pdf). This dataset represents a much finer spatial resolution of climate model (2.2km grid) than typical climate-models, representing a step forward in the ability to simulate small scale behavior (in particular 'atmospheric convection'), and the influence of mountains, coastlines and urban areas. As a result, the CPM provides access to credible climate information important for small-scale weather features and also on local (kilometre) scale; which is particularly important for improving our understanding of climate change in cities. -# Why Bias Correction? +The UKCP18-CPM represents a high-emission scenario (RCP 8.5). -Regional climate models contain systematic errors, or biases in their output [^1]. Biases arise in RCMs for a number of reasons, such as the assumptions in the general circulation models (GCMs), and in the downscaling process from GCM to RCM. +The UKCP18-CPM is comprised of 12 ensemble members (or runs), driven by the same 12km Regional Climate Model (Strand 3 12km RCM ensemble). In addition to run 1, we selected the following runs: -Researchers, policy-makers and other stakeholders wishing to use publicly available RCMs need to consider a range of "bias correction” methods (sometimes referred to as "bias adjustment" or "recalibration"). Bias correction methods offer a means of adjusting the outputs of RCM in a manner that might better reflect future climate change signals whilst preserving the natural and internal variability of climate [^2]. +- Run 05: Represents the ensemble member with the second lowest mean annual tasmax of all ensembles members +- Run 06: Represents the ensemble member with the second highest mean annual tasmax of all ensembles members +- Run 07 & Run 08: Represent the ensemble members with the average mean annual tasmax of all ensemble members -Part of the `clim-recal` project is to review several bias correction methods. This work is ongoing and you can find our initial [taxonomy here](https://docs.google.com/spreadsheets/d/18LIc8omSMTzOWM60aFNv1EZUl1qQN_DG8HFy1_0NdWk/edit?usp=sharing). When we've completed our literature review, it will be submitted for publication in an open peer-reviewed journal. +We believe that this combination will provide users with enough uncertainty in their estimates to appropriately assess bias correction methods. -Our work is however, just like climate data, intended to be dynamic, and we are in the process of setting up a pipeline for researchers creating new methods of bias correction to be able to submit their methods for inclusion on in the `clim-recal` repository. -[^1]: Senatore et al., 2022, -[^2]: Ayar et al., 2021, +## HADS +[HadUK-Grid](https://www.metoffice.gov.uk/research/climate/maps-and-data/data/haduk-grid/haduk-grid) is a comprehensive collection of climate data for the UK, compiled from various land surface observations across the country. This data is organized into a uniform grid to ensure consistent coverage throughout the UK at up to 1km x 1km resolution. The dataset, spanning from 1836 to the present, includes a variety of climate variables such as air temperature, precipitation, sunshine, and wind speed, available on daily, monthly, seasonal, and annual timescales. # Contributing -If you have suggestions on the repository, or would like to include a new method (see below) or library, please +If you have suggestions on the repository, please: - raise an [issue](https://github.com/alan-turing-institute/clim-recal/issues) - [get in touch](mailto:clim-recal@turing.ac.uk) -- see our [contributing](docs/contributing.md) section, which includes details on contriubting to the documentation. +- see our [contributing](docs/contributing.md) section, which includes details on contributing to the documentation. All are welcome and appreciated. -# Future plans -- **Finish refactor for BC**: The infrastructure for testing bias correction methods needs some reworking and documentation. -- **Release BC results**: Provide results from example BC runs. -- **More BC Methods**: Further bias correction of UKCP18 products. *This is planned for a future release and is not available yet.* -- **Pipeline for adding new methods**: *This is planned for a future release and is not available yet.* - - ## Acknowledgements Prior to 12th September 2024 we included a reference to the [python-cmethods](https://github.com/btschwertfeger/python-cmethods) library, written by Benjamin Thomas Schwertfeger. @@ -102,6 +102,9 @@ Inadvertently, we did not identify that the license for the `python-cmethods` li * Added the citation below. -## Citation +## Citations + +[^1]: Senatore et al., 2022, +[^2]: Ayar et al., 2021, **python-cmethods**: Benjamin T. Schwertfeger. (2024). btschwertfeger/python-cmethods: v2.3.0 (v2.3.0). Zenodo. https://doi.org/10.5281/zenodo.12168002 diff --git a/_quarto.yml b/_quarto.yml index 8df764c8..a140ae70 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -9,9 +9,6 @@ project: - "README.md" - "setup-instructions.md" - "!clim-recal.Rproj" - - "R/README.md" - - "R/misc/Identifying_Runs.md" - - "R/comparing-r-and-python/HADs-reprojection/WIP-Comparing-HADs-grids.md" - "docs/cpm_projection.qmd" - "docs/reference" - "docs/contributing.md" diff --git a/docs/download.qmd b/docs/download.qmd index 4ebe3a43..069623ea 100644 --- a/docs/download.qmd +++ b/docs/download.qmd @@ -38,13 +38,13 @@ For a given region `` (either `Scotland`, `Glasgow`, `Manchester` or `Lo ```shell grep -iE "crop.*hads.*.*.*_[0-9]{8}-[0-9]{8}.*" data-v1.0.txt | xargs -n 1 curl -O; gunzip *.nc.gz ``` -For example, for region is `Manchester`, measure is `tasmax`: +For example, for region is `manchester`, measure is `tasmax`: ```shell grep -iE ".*crop.*hads.*manchester.*tasmax.*_[0-9]{8}-[0-9]{8}\.nc\.gz" data-v1.0.txt | xargs -n 1 curl -O; gunzip *.nc.gz ``` ### CPM -For a given region `` (either `Scotland`, `Glasgow`, `Manchester` or `London`), for measurement `` (either `tasmax`, `tasmin` or `pr`), for run `` (either `01`, `05`, `06`, `07`, `08`), the yearly data can be downloaded and decompressed with: +For a given region `` (either `Scotland`, `Glasgow`, `Manchester` or `London`), for measurement `` (either `tasmax`, `tasmin` or `pr`), for run `` (either `01`, `05`, `06`, `07`, `08`), the yearly data can be downloaded and decompressed with: ```shell grep -iE "crop.*cpm.*.*.*_[0-9]{8}-[0-9]{8}.*" data-v1.0.txt | xargs -n 1 curl -O; gunzip *.nc.gz ```