Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add: Introduction for users new to Python geospatial #242

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add: Introduction for users new to Python geospatial #242

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d9e86dc
wip introduction
m-cappi Dec 20, 2022
e03edac
wip hub introduction
m-cappi Dec 21, 2022
4c7f7b6
wip hub introduction
m-cappi Dec 22, 2022
5359d29
hub introduction revision
m-cappi Dec 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
224 changes: 224 additions & 0 deletions quickstarts/introduction.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "04d77799-afa8-4f49-9c59-93a5b358d05b",
"metadata": {
"tags": []
},
"source": [
"## Warming up: what will you need going forward with Planetary Computer Hub?\n",
"\n",
"The Planetary Computer Hub development environment relies on open-source tools to work with the data. In our effort to welcome everyone who may not be so familiarized on using Python tools for academic research, we present an overview on those and some reference guides."
]
},
{
"cell_type": "markdown",
"id": "9457a143-c059-4a5e-9240-2d6d51bb029b",
"metadata": {
"tags": []
},
"source": [
"### Tools landscape"
]
},
{
"cell_type": "markdown",
"id": "555ea09d-1b57-4442-8b11-24424408ffb6",
"metadata": {},
"source": [
"### Retrieving collections"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e7970c2b-0ec5-444e-aedc-4ab054e2853f",
"metadata": {},
"source": [
"#### PySTAC\n",
"\n",
"[PySTAC](https://pystac.readthedocs.io/en/latest/) is a library for working with [SpatioTemporal Asset Catalogs](https://stacspec.org/) (STAC) in Python 3.\n",
"On Planetary Computer, the main purpose behind PySTAC is to serve as an efficient crawler of STAC catalogs.\n",
"It enables us to manipulate the metadata on local, prior to loading the assets.\n",
"\n",
"A comprehensive example of how it is used on Planetary Computer can be found in the [Reading Data from the STAC API](../quickstarts/reading-stac.ipynb) quickstart.\n",
"\n",
"For getting the specifics on the PySTAC library, refer to the [official documentation](https://pystac.readthedocs.io/en/latest/)."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "76362f7b-6dd0-4320-b9e5-5ef48a15299e",
"metadata": {},
"source": [
"#### PySTAC-client\n",
"> [PySTAC-Client](https://pystac-client.readthedocs.io/en/latest/index.html#) builds upon PySTAC library to add support for STAC APIs in addition to static STAC catalogs.\n",
"\n",
"On Planetary Computer, [PySTAC-Client](https://pystac-client.readthedocs.io/en/latest/index.html#) performs a key integration point with Planetary Computer's STAC API by abstracting the queries and filters when working with the API.\n",
"This way, we can consume and search through the API based on metadata to locate only the collection/s we are interested in.\n",
"\n",
"An explanation on consuming the Planetary Computer API and how it translates to using PySTAC-Client can be found on the [Using the Planetary Computer’s Data API](../quickstarts/using-the-data-api.ipynb) quickstart.\n",
"\n",
"For getting the specifics on the PySTAC-Client library, refer to the [official documentation](https://pystac-client.readthedocs.io/en/latest/index.html)."
]
},
{
"cell_type": "markdown",
"id": "33d91f71-69cb-4274-81c8-7a2f690daea4",
"metadata": {
"tags": []
},
"source": [
"### Data manipulation"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "6de769e9-be65-4fbc-9c9d-527ae7f85001",
"metadata": {},
"source": [
"#### Pandas\n",
"\n",
"[Pandas](https://pandas.pydata.org/docs/index.html) is the go-to library for handling [tabular data](https://en.wikipedia.org/wiki/Table_(information)) in a neat, consistent and programmatic way.\n",
"It's a great library loaded with data analysis and manipulation tools.\n",
"\n",
"An example of reading tabular data using Pandas can be found in the [Reading Tabular Data](https://planetarycomputer.microsoft.com/docs/quickstarts/reading-tabular-data/) quickstart.\n",
"Another quick example on how to read any STAC asset, stored in JSON format, using Pandas can be found in the [Bulk STAC item queries with GeoParquet](../quickstarts/stac-geoparquet.ipynb#expanding-nested-fields) quickstart.\n",
"\n",
"[Project Pythia's introduction to Pandas](https://foundations.projectpythia.org/core/pandas/pandas.html) covers the basics of what you will need, such as slicing and performing an exploratory analysis on _DataFrames_ and _DataSeries_.\n",
"But the [Pandas official documentation](https://pandas.pydata.org/docs/index.html) is thorough and leaves nothing to be desired."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "64819e76-71cf-45ae-9e16-2b0e0a374b87",
"metadata": {},
"source": [
"#### GeoPandas\n",
"\n",
"> [GeoPandas](https://geopandas.org/en/stable/index.html), as the name suggests, extends the popular data science library pandas by adding support for geospatial data.\n",
"\n",
"The reason we choose to use GeoPandas is that it allows us to handle points, lines, curves and polygons (a.k.a., [vectors](https://en.wikipedia.org/wiki/Vector_graphics)). \n",
"Then we can reference these shapes, packed with their own associated data, to a [Coordinate Reference System](https://en.wikipedia.org/wiki/Spatial_reference_system) (CRS). \n",
"Under CRS, you will often find different [EPSG](https://en.wikipedia.org/wiki/EPSG_Geodetic_Parameter_Dataset) codes which represent different projections of our planet. \n",
"\n",
"GeoPandas is often used for mapping areas of interest, such as regions, cities, forests or lakes.\n",
"One such example can be found in the [Bulk STAC item queries with GeoParquet](../quickstarts/stac-geoparquet.ipynb) quickstart.\n",
"Dots or smaller areas can also be used to track objects or natural phenomena across time.\n",
"An example for this kind of usage is the [Visualizing Hurricane Florence](../tutorials/hurricane-florence-animation.ipynb) tutorial.\n",
"\n",
"For getting the specifics on the StackSTAC Geopandas, refer to the [official documentation](https://geopandas.org/en/stable/index.html).\n",
"\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cc5c4390-fbbd-4783-b4c9-ffdec2c80813",
"metadata": {
"tags": []
},
"source": [
"#### Xarray\n",
"\n",
"[Xarray](https://docs.xarray.dev/en/stable/index.html) is our tool of choice for dealing with images and other [raster](https://en.wikipedia.org/wiki/Raster_graphics) data sources.\n",
"\n",
"> **[Why Xarray?](https://docs.xarray.dev/en/stable/getting-started-guide/why-xarray.html)**\n",
"> \n",
"> Xarray introduces labels in the form of dimensions, coordinates and attributes on top of raw NumPy-like multidimensional arrays, which allows for a more intuitive, more concise, and less error-prone developer experience.\n",
"\n",
"Xarray borrows heavily from Pandas, which enhances our human productivity by setting us up with a wide-spread interface. \n",
"Also, it tightly integrates with Dask which in turn enhances our computing power through parallel processing.\n",
"\n",
"A quick example on using Xarray can be found on the [Reading Zarr Data](../quickstarts/reading-zarr-data.ipynb) quickstart.\n",
"\n",
"For those looking for an introduction on _DataArrays_ and _DataSets_ with Xarray, we recommend heading to [Project Pythia's introduction to Xarray](https://foundations.projectpythia.org/core/xarray/xarray-intro.html) given its bias towards science. \n",
"Nonetheless, it's always good to keep in mind the [Xarray's official documentation](https://docs.xarray.dev/en/stable/index.html) for browsing its full spectrum."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f2a408d2",
"metadata": {},
"source": [
"#### StackSTAC\n",
"\n",
"[StackSTAC](https://stackstac.readthedocs.io/en/latest/index.html) is a library that can be leveraged to easily transform a PySTAC item's assets into a Xarray DataArray with some extra magic along the way:\n",
"\n",
"- Figure out the geospatial parameters from the PySTAC metadata\n",
"- Transfer PySTAC metadata into Xarray coordinates for easy indexing\n",
"- Bootstrap the DataArray to be lazy and harness the computing boost provided by Dask\n",
"\n",
"A quick example of how it is used on Planetary Computer can be found in the [Cloudless Mosaic](../tutorials//cloudless-mosaic-sentinel2.ipynb) tutorial.\n",
"\n",
"For getting the specifics on the StackSTAC library, refer to the [official documentation](https://stackstac.readthedocs.io/en/latest/index.html).\n"
]
},
{
"cell_type": "markdown",
"id": "f1effa74-ee1c-4448-b4a7-5b2a0057c7f1",
"metadata": {},
"source": [
"### Computing power"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8f941ffc-5706-449c-9059-bac5f6bf14d4",
"metadata": {},
"source": [
"#### Dask\n",
"\n",
"[Dask](https://www.dask.org/get-started) helps us process big chunks of data by breaking it apart into smaller pieces.\n",
"A fundamental characteristic of Dask is that it is lazily evaluated, and there are two key things to take away from this:\n",
"- The results won't be computed until we call on `*.compute()`\n",
"- We can get a preview of the data structure and type\n",
"\n",
"> **[Briefly, what problem does Dask solve for us?](https://docs.dask.org/en/stable/faq.html#briefly-what-problem-does-dask-solve-for-us)**\n",
"> \n",
"> Dask is a general purpose parallel programming solution. As such it is used in many different ways.\n",
">\n",
"> However, the most common problem that Dask solves is connecting Python analysts to distributed hardware, particularly for data science and machine learning workloads. [...]\n",
"\n",
"See [Scale With Dask](../quickstarts/scale-with-dask.ipynb) for more on using Dask.\n",
"For getting the specifics on the Dask library, refer to the [official documentation](https://docs.dask.org/en/stable/).\n",
"\n",
"\n",
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)]"
},
"vscode": {
"interpreter": {
"hash": "ed1b328372cf0187c2b7d52ec5a4e52f1e11ef76628e78d3a5124a59986bd55e"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}