Skip to content

Commit

Permalink
fix(doc): fix syntax errors in WIP-Comparing-HADs-grids.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@spool
spool committed Feb 15, 2024
1 parent b2cb624 commit 2fb76c3
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 38 deletions.
28 changes: 7 additions & 21 deletions R/comparing-r-and-python/HADs-reprojection/WIP-Comparing-HADs-grids.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ doc is to:

### **1a. HadUK grid resampled in R**

Resampling script [here](https://github.com/alan-turing-institute/clim-recal/blob/main/R/Resampling.HADs.inR.R)
The 2.2km grid was derived from a reprojected (to BNG) UKCP 2.2km .nc file
Resampling script [here](https://github.com/alan-turing-institute/clim-recal/blob/main/R/Resampling.HADs.inR.R)
The 2.2km grid was derived from a reprojected (to BNG) UKCP 2.2km .nc file

In resampling it resampled the Sea as xx so replacing those vals as NA

Expand All @@ -52,11 +52,8 @@ r1[r1 > 200] = NA

#check the crs
crs(r1, proj=T)
```

## [1] "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +a=6377563.396 +rf=299.324961266495 +units=m +no_defs"
## [1] "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +a=6377563.396 +rf=299.324961266495 +units=m +no_defs"

``` r
#Plot to check
plot(r1$tasmax_1)
```
Expand Down Expand Up @@ -99,11 +96,9 @@ og <- paste0(f, hads.tasmax2)

og <- rast(og)
crs(og, proj=T)
```

## [1] "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +a=6377563.396 +rf=299.324961266495 +units=m +no_defs"
## [1] "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +a=6377563.396 +rf=299.324961266495 +units=m +no_defs"

``` r
plot(og$tasmax_1)
```

Expand All @@ -118,11 +113,9 @@ ukcp <- rast(f)
ukcp.r <- ukcp$`tasmax_rcp85_land-cpm_uk_2.2km_01_day_19991201-20001130_31`

crs(ukcp.r, proj=T)
```

## [1] "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +units=m +no_defs"
## [1] "+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +units=m +no_defs"

``` r
#plot(ukcp.r)
```

Expand All @@ -146,11 +139,8 @@ b <- Sys.time()
r1_c <- terra::crop(r1, scotland, snap="in")
e <- Sys.time()
e-b
```

## Time difference of 0.02198005 secs

``` r
## Time difference of 0.02198005 secs
plot(r1_c$tasmax_1)
```

Expand All @@ -161,11 +151,7 @@ b <- Sys.time()
r2_c <- terra::crop(r2, scotland, snap="in")
e <- Sys.time()
e-b
```

## Time difference of 33.57785 secs

``` r
## Time difference of 33.57785 secs
plot(r2_c$tasmax_1)
```

Expand Down
42 changes: 25 additions & 17 deletions docs/pipeline_guidance.qmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,23 @@ This is a detailed guide to our analysis pipeline.

## Prerequisites

For our bias correction methods, we tap into dedicated packages in both Python and R ecosystems. The integration of these languages allows us to utilize the cutting-edge functionalities implemented in each.
For our bias correction methods, we tap into dedicated packages in both `python` and `R` ecosystems. The integration of these languages allows us to utilize the cutting-edge functionalities implemented in each.

Given this dual-language nature of our analysis pipeline, we also provide preprocessing scripts written in both Python and R. To facilitate a seamless experience, users are required to set up both Python and R environments as detailed below.
Given this dual-language nature of our analysis pipeline, we also provide preprocessing scripts written in both `python` and `R`. To facilitate a seamless experience, users are required to set up both `python` and `R` environments as detailed below.

### Setting up your R environment
### Setting up your `R` environment

#### Download and Install R
#### Download and Install `R`

Visit [CRAN (The Comprehensive R Archive Network)](https://cran.r-project.org/) to download the latest version of R compatible with your operating system. Then verify successful installation via command line:
Visit [CRAN (The Comprehensive R Archive Network)](https://cran.r-project.org/) to download the latest version of `R` compatible with your operating system. Then verify successful installation via command line:

```{bash}
R --version
```

#### Install Necessary R Packages
#### Install Necessary `R` Packages

Our analysis utilizes several R packages. You can install these packages by starting R (just type `R` in your command line and press enter) and entering the following commands in the R console:
Our analysis utilizes several `R` packages. You can install these packages by starting `R` (just type `R` in your command line and press enter) and entering the following commands in the `R` console:

```{R}
#| eval: false
Expand All @@ -37,19 +37,27 @@ install.packages("package2")
#... (continue for all necessary packages)
```

Replace `"package1"`, `"package2"`, etc., with the actual names of the required packages. A list of required R packages is provided in the 'R_Package_Requirements.txt' file.
Replace `"package1"`, `"package2"`, etc., with the actual names of the required packages. A list of required `R` packages is usually at the top of each `.R`, `.Rmd`, some `R` specific `.md` files. Below is an example from [WIP Comparing HADs grids](../R/comparing-r-and-python/HADs-reprojection/WIP-Comparing-HADs-grids.html):

### Setting up your python environment
```{r}
#| eval: false
library(terra)
library(sp)
library(exactextractr)
```

For your python environment, we provide an Anaconda environment file for ease-of-use.
### Setting up your `python` environment

For your `python` environment, we provide an Anaconda environment file for ease-of-use.
```{bash}
#| eval: false
conda env create -f environment.yml
```

::: {.callout-warning}
To reproduce our exact outputs, you will require GDAL version 3.4. Please be aware that this specific version of GDAL requires a different Python version than the one specified in our environment file. Therefore, we have not included it in the environment file and instead, for the reprojection step, you'll need to install GDAL (for example using conda) and set up a new environment:
To reproduce our exact outputs, you will require GDAL version 3.4. Please be aware that this specific version of GDAL requires a different `python` version than the one specified in our environment file. Therefore, we have not included it in the environment file and instead, for the reprojection step, you'll need to install GDAL (for example using `conda`) and set up a new environment:
```{bash}
#| eval: false
Expand All @@ -68,7 +76,7 @@ parallel --version

### The `cmethods` library

This repository contains a python script used to run debiasing in climate data using a fork of the [original python-cmethods](https://github.com/btschwertfeger/python-cmethods) module written by Benjamin Thomas Schwertfeger's , which has been modified to function with the dataset used in the clim-recal project. This library has been included as a submodule to this project, so you must run the following command to pull the submodules required.
This repository contains a `python` script used to run debiasing in climate data using a fork of the [original `python-cmethods`](https://github.com/btschwertfeger/python-cmethods) module written by Benjamin Thomas Schwertfeger's , which has been modified to function with the dataset used in the `clim-recal` project. This library has been included as a submodule to this project, so you must run the following command to pull the submodules required.

```{bash}
git submodule update --init --recursive
Expand All @@ -80,7 +88,7 @@ git submodule update --init --recursive

This streamlined pipeline is designed for raw data provided by the Met Office, accessible through the [CEDA archive]((https://catalogue.ceda.ac.uk/uuid/ad2ac0ddd3f34210b0d6e19bfc335539)). It utilizes [UKCP](https://data.ceda.ac.uk/badc/ukcp18/data/land-cpm/uk/2.2km) control, scenario data at 2.2km resolution, and [HADs](https://data.ceda.ac.uk/badc/ukmo-hadobs/data/insitu/MOHC/HadOBS/HadUK-Grid/v1.1.0.0/1km) observational data. For those unfamiliar with this data, refer to our [the dataset](#the-dataset) section.

To access the data,[register here]((https://archive.ceda.ac.uk/)) at the CEDA archive and configure your FTP credentials in "My Account". Utilize our [ceda_ftp_download.py](../python/data_download/ceda_ftp_download.py) script to download the data.
To access the data, [register here](https://archive.ceda.ac.uk/) at the CEDA archive and configure your FTP credentials in "My Account". Utilize our [ceda_ftp_download.py](../python/data_download/ceda_ftp_download.py) script to download the data.

```{bash}
#| eval: false
Expand Down Expand Up @@ -115,7 +123,7 @@ The first step in our analysis pipeline is to reproject the UKCP datasets to the

::: {.callout-warning}
Note that, to reproduce our exact pipeline, we switch environments here as explained in the requirements.
```bash
```{bash}
#| eval: false
conda activate gdal_env
Expand All @@ -133,7 +141,7 @@ sh reproject_all.sh path_to_netcdf_files

## Resampling the data

Resample the HADs UK dataset from 1km to 2.2km grid to match the UKCP reprojected grid. We run the resampling python script specifying the `--input` location of the reprojected files from the previous step, the UKCP `--grid` file an the `--output` location for saving the resampled files.
Resample the HADs UK dataset from 1km to 2.2km grid to match the UKCP reprojected grid. We run the resampling `python` script specifying the `--input` location of the reprojected files from the previous step, the UKCP `--grid` file an the `--output` location for saving the resampled files.

```{bash}
#| eval: false
Expand Down Expand Up @@ -179,11 +187,11 @@ The preprocess_data.py script also aligns the calendars of the historical simula
## Applying the bias correction

::: {.callout-note}
By March 2023 we have only implemented the [python-cmethods](https://github.com/alan-turing-institute/python-cmethods) library.
By March 2023 we have only implemented the [`python-cmethods`](https://github.com/alan-turing-institute/python-cmethods) library.
:::


The [run_cmethods.py](../python/debiasing/run_cmethods.py) allow us to adjusts climate biases in climate data using the python-cmethods library. It takes as input observation data (HADs data), control data (historical UKCP data), and scenario data (future UKCP data), and applies a correction method to the scenario data. The resulting output is saved as a `.nc` to a specified directory.
The [run_cmethods.py](../python/debiasing/run_cmethods.py) allow us to adjusts climate biases in climate data using the `python-cmethods` library. It takes as input observation data (HADs data), control data (historical UKCP data), and scenario data (future UKCP data), and applies a correction method to the scenario data. The resulting output is saved as a `.nc` to a specified directory.

The script will also produce a time-series and a map plot of the debiased data. To run this you need to replace `path_to_validation_data` with the output directories of the previous step and specify `path_to_corrected_data` as your output directory for the bias corrected data. You can also specify your preferred `bias_correction_method` (e.g. quantile_delta_mapping).

Expand Down

0 comments on commit 2fb76c3

Please sign in to comment.