Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@ptaconet
ptaconet committed Aug 29, 2024
1 parent ce29ea3 commit 984bf76
Show file tree
Hide file tree
Showing 2 changed files with 321 additions and 804 deletions.
55 changes: 27 additions & 28 deletions README.Rmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ library(dplyr)

<!-- badges: start -->
[![licence](https://img.shields.io/badge/Licence-GPL--3-blue.svg)](https://www.r-project.org/Licenses/GPL-3)
[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/modisfast)](https://cran.r-project.org/package=modisfast)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/modisfast)](https://cran.r-project.org/package=modisfast)
[![Github_Status_Badge](https://img.shields.io/badge/Github-0.9.2-blue.svg)](https://github.com/ptaconet/modisfast)
[![SWH](https://archive.softwareheritage.org/badge/origin/https://github.com/ptaconet/modisfast/)](https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/ptaconet/modisfast)
[![R-CMD-check](https://github.com/ptaconet/modisfast/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/ptaconet/modisfast/actions/workflows/R-CMD-check.yaml)
Expand Down Expand Up @@ -50,19 +50,19 @@ This package enables to build OPeNDAP (https) URLs given input parameters such a
• <a href="#collections-available-in-modisfast">Data collections available</a><br>
• <a href="#foundational-framework">Foundational framework </a><br>
• <a href="#comparison-with-similar-r-packages">Comparison with similar R packages</a><br>
• <a href="#future-developments">Future developments</a><br>
• <a href="#citation">Citation</a><br>
• <a href="#future-developments">Future developments</a><br>
• <a href="#contributing">Contributing</a><br>
• <a href="#acknowledgments">Acknowledgments</a><br>
</p>

## Overview

**`modisfast`** is an R package designed for **easy** and **fast** downloads and import of some widely-used satellite-derived environmental data, including [**MODIS**](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/) Land products, [**VIIRS**](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/) Land products, and [**GPM**](https://gpm.nasa.gov/data/) products (Global Precipitation Measurement Mission).
**`modisfast`** is an R package designed for **easy** and **fast** downloads of [**MODIS**](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/) Land products, [**VIIRS**](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/) Land products, and [**GPM**](https://gpm.nasa.gov/data/) (Global Precipitation Measurement Mission) Earth Observation data.

**`modisfast`** uses the abilities offered by the [OPeNDAP](https://www.opendap.org/about/) framework (*Open-source Project for a Network Data Access Protocol*) to download a subset of Earth science data cube, along spatial, temporal or any other data dimension (depth, ...). This way, it reduces downloading time and disk usage to their minimum : no more 1° x 1° MODIS tiles with 10 bands when your region of interest is only 30 km x 30 km wide and you need 2 bands ! Moreover, `modisfast` enables parallel downloads of data.
`modisfast` uses the abilities offered by the [OPeNDAP](https://www.opendap.org/about/) framework (*Open-source Project for a Network Data Access Protocol*) to download a subset of Earth Observation data cube, along spatial, temporal or any other data dimension (depth, ...). This way, it reduces downloading time and disk usage to their minimum : no more 1° x 1° MODIS tiles with 10 bands when your region of interest is only 30 km x 30 km wide and you need 2 bands ! Moreover, `modisfast` enables parallel downloads of data.

`modisfast` is hence particularly suited for retrieving MODIS or VIIRS data **over long time series** and **over areas**, rather than short time series and points.
This package is hence particularly suited for retrieving MODIS or VIIRS data **over long time series** and **over areas**, rather than short time series and points.

Importantly, the robust, sustainable, and cost-free [foundational framework](#foundational-framework) of `modisfast`, both for the data provider (NASA) and the software (R, OPeNDAP, the `tidyverse` and `GDAL` suite of packages and software), guarantees the long-term reliability and open-source nature of the package.

Expand Down Expand Up @@ -100,7 +100,7 @@ devtools::install_github("ptaconet/modisfast")

## Get Started

Accessing and opening MODIS data with `modisfast` is a simple 3-steps workflow, as shown in the example below. This example shows how to download a one-year-long monthly time series of MODIS Normalized Difference Vegetation Index (NDVI) at 1 km spatial resolution over the whole country of Madagascar.
Accessing and opening MODIS data with `modisfast` is a simple 3-steps workflow. This example shows how to download and import a one-year-long monthly time series of MODIS Normalized Difference Vegetation Index (NDVI) at 1 km spatial resolution over the whole country of Madagascar.

**1/ First, define the variables of interest (ROI, time frame, collection, and bands) :**

Expand Down Expand Up @@ -142,7 +142,7 @@ res_dl <- mf_download_data(urls, parallel = T)
```

**3/ And finally, open the data in R as a `terra::SpatRaster` object using the function `mf_import_data()`** ( :warning: see [here](https://ptaconet.github.io/modisfast/articles/get_started.html#warning-import) why you should use this function, instead of the original `terra::rast()`, in the context of `modisfast`) :
**3/ And finally, import the data in R as a `terra::SpatRaster` object using the function `mf_import_data()`** ( :warning: see [here](https://ptaconet.github.io/modisfast/articles/get_started.html#warning-import) why you should use this function, instead of the original `terra::rast()`, in the context of `modisfast`) :

```{r , eval = F, message=F}
Expand Down Expand Up @@ -184,18 +184,15 @@ Apart from these performance aspects, ethical considerations have driven the dev
We thank in advance people that use `modisfast` for citing it in their work / publication(s). For this, please use the citation provided at this link [zenodo link to add] or through `citation("modisfast")`.
-->

## Collections available in modisfast
## Collections available in `modisfast`

Currently `modisfast` supports download of `r nrow(mf_list_collections())` data collections, extracted from the following meta-collections :

* [MODIS land products](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/) made available by the [NASA / USGS LP DAAC](https://lpdaac.usgs.gov/) ( :arrow_right: [source OPeNDAP server](https://opendap.cr.usgs.gov/opendap/hyrax/)) ;
* [VIIRS land products](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/) made available by the [NASA / USGS LP DAAC](https://lpdaac.usgs.gov/) ( :arrow_right: [source OPeNDAP server](https://opendap.cr.usgs.gov/opendap/hyrax/))

In addition, `modisfast` supports download of the following satellite-derived environmental data :

* [Global Precipitation Measurement](https://gpm.nasa.gov/missions/GPM) (GPM) made available by the [NASA / JAXA GES DISC](https://disc.gsfc.nasa.gov/) ( :arrow_right: [source OPeNDAP server](https://gpm1.gesdisc.eosdis.nasa.gov/opendap/hyrax/GPM_L3/)).

Details of each product available for download are provided in the tables below or through the function `mf_list_collections()`. Want more details on a specific collection ? Click on the "DOI" column !
Details of each product available for download are provided in the tables below or through the function `mf_list_collections()`.

<details><summary><b>MODIS and VIIRS data collections accessible with modisfast (click to expand)</b></summary>
<p>
Expand All @@ -204,10 +201,11 @@ Details of each product available for download are provided in the tables below
library(kableExtra)
mf_list_collections() %>%
dplyr::filter(source %in% c("MODIS","VIIRS")) %>%
dplyr::select(collection,long_name,source,nature,doi,url_programmatic_access) %>%
dplyr::rename(Collection=collection,Nature=nature,Name=long_name,DOI=doi,Opendap_server=url_programmatic_access,Source=source) %>%
kable() %>%
kable_styling()
dplyr::mutate(collection = cell_spec(collection, "html", link = doi)) %>%
dplyr::select(collection,source,long_name,nature) %>%
dplyr::rename(Collection=collection,Type=nature,Name=long_name,Source=source) %>%
kable("html", escape = FALSE) %>%
kable_styling(bootstrap_options = c("hover", "condensed"))
```

</p>
Expand All @@ -219,19 +217,20 @@ mf_list_collections() %>%
```{r, eval = TRUE, echo=F,message=F}
library(kableExtra)
mf_list_collections() %>%
dplyr::filter(!(source %in% c("MODIS","VIIRS"))) %>%
dplyr::select(collection,long_name,source,nature,doi,url_programmatic_access) %>%
rename(Collection=collection,Nature=nature,Name=long_name,DOI=doi,Opendap_server=url_programmatic_access,Source=source) %>%
kable() %>%
kable_styling()
dplyr::filter(!(source %in% c("MODIS","VIIRS"))) %>%
dplyr::mutate(collection = cell_spec(collection, "html", link = doi)) %>%
dplyr::select(collection,source,long_name,nature) %>%
dplyr::rename(Collection=collection,Type=nature,Name=long_name,Source=source) %>%
kable("html", escape = FALSE) %>%
kable_styling(bootstrap_options = c("hover", "condensed"))
```

</p>
</details>

## Foundational framework {#foundational-framework}

Technically, `modisfast` is a programmatic interface (R wrapper) to several NASA OPeNDAP (https://www.opendap.org/) servers. OPeNDAP is the acronym for *Open-source Project for a Network Data Access Protocol* and designates both the software, the access protocol, and the corporation that develops them. The OPeNDAP is designed to simplify access to structured and high-volume data, such as satellite products, over the Web. It is a collaborative effort involving multiple institutions and companies, with open-source code, free software, and adherence to the [Open Geospatial Consortium](https://www.ogc.org/) (OGC) standards. It is widely used by NASA, which partly finances it.
Technically, `modisfast` is a programmatic interface (R wrapper) to several NASA [OPeNDAP](https://www.opendap.org/) servers. OPeNDAP is the acronym for *Open-source Project for a Network Data Access Protocol* and designates both the software, the access protocol, and the corporation that develops them. The OPeNDAP is designed to simplify access to structured and high-volume data, such as satellite products, over the Web. It is a collaborative effort involving multiple institutions and companies, with open-source code, free software, and adherence to the [Open Geospatial Consortium](https://www.ogc.org/) (OGC) standards. It is widely used by NASA, which partly finances it.

A key feature of OPeNDAP is its capability to apply filters at the data download process, ensuring that only the necessary data is retrieved. These filters, specified within a URL, can be spatial, temporal, or dimensional. Although powerful, OPeNDAP URLs are not trivial to build. `modisfast` facilitates this process by constructing the URL based on the spatial, temporal, and dimensional filters provided by the user in the function `mf_get_url()`.

Expand All @@ -241,7 +240,7 @@ The following URL :arrow_down:

https<nolink>://opendap.cr.usgs.gov/opendap/hyrax/MOD11A2.061/h17v08.ncml.nc4?MODIS_Grid_8Day_1km_LST_eos_cf_projection,LST_Day_1km[775:793][55:140][512:560],LST_Night_1km[775:793][55:140][512:560],QC_Day[775:793][55:140][512:560],QC_Night[775:793][55:140][512:560],time[775:793],YDim[55:140],XDim[512:560]

provides a link to download the [MOD11A2.061](https://doi.org/10.5067/MODIS/MOD11A2.061) data in netCDF, subsetted for :
is a link to download the following subset of MOD11A2.061 data in netCDF :

- bands LST_Day_1km, LST_Night_1km, QC_Day, QC_Night ;
- each available date between the 2017-01-01 and the 2017-06-01 ;
Expand All @@ -255,17 +254,13 @@ These OPeNDAP URLs are not trivial to build. `modisfast` converts the spatial, t

There are other R packages available for accessing MODIS data. These include :

* [`appeears`](https://github.com/bluegreen-labs/appeears)
* [`MODIS`](https://github.com/fdetsch/MODIS)
* [`MODIStsp`](https://github.com/ropensci/MODIStsp)
* [`MODIStools`](https://github.com/ropensci/MODIStools)
* [`appeears`](https://github.com/bluegreen-labs/appeears)

Take a look at the article ["Comparison of performance with other similar R packages"](https://ptaconet.github.io/modisfast/articles/perf_comp.html) to get an overview of how `modisfast` compares to these packages in terms of data access time.

## Future developments

Future developments of the package may include access to additional data collections from other OPeNDAP servers, and support for a variety of data formats as they become available from data providers through their OPeNDAP servers. Furthermore, the creation of an RShiny application on top of the package is being considered, as a means of further simplifying data access for users with limited coding skills.

## Citation

This package is licensed under a [GNU General Public License v3.0 or later](https://www.gnu.org/licenses/gpl-3.0-standalone.html) license.
Expand All @@ -274,6 +269,10 @@ We thank in advance people that use `modisfast` for citing it in their work / pu

> Taconet, P. & Moiroux N.(2024). modisfast: Fast and Efficient Access to MODIS Earth Observation Data. In CRAN: Contributed Packages. The R Foundation. https://doi.org/10.32614/cran.package.modisfast
## Future developments

Future developments of the package may include access to additional data collections from other OPeNDAP servers, and support for a variety of data formats as they become available from data providers through their OPeNDAP servers. Furthermore, the creation of an RShiny application on top of the package is being considered, as a means of further simplifying data access for users with limited coding skills.

## Contributing

All types of contributions are encouraged and valued. For more information, check out our [Contributor Guidelines](https://github.com/ptaconet/modisfast/blob/main/CONTRIBUTING.md).
Expand Down
Loading

0 comments on commit 984bf76

Please sign in to comment.