notebooks/02-lines.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d7870705",
   "metadata": {},
   "source": [
    "# Lines\n",
    "\n",
    "## Day 2 - 30 Day Map Challenge\n",
    "\n",
    "For this map I wanted to demonstrate how to use the [Wherobots Open Data Catalog](https://docs.wherobots.services/1.2.0/tutorials/opendata/introduction/#open-data-catalog) with [SedonaDB](https://wherobots.com/sedona-db/) to use the [Overture Maps transportation dataset](https://medium.com/@dr.jiayu/harnessing-overture-maps-data-apache-sedonas-journey-from-parquet-to-geoparquet-d99f7767a499). Specifically we'll look at roads under construction throughout the world.\n",
    "\n",
    "\n",
    "![](../img/02-lines-final.png)\n",
    "\n",
    "You can follow along by creating a free account on [Wherobots Cloud.](https://www.wherobots.services)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "93dd4712",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sedona.spark import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd9b4b27",
   "metadata": {},
   "outputs": [],
   "source": [
    "config = SedonaContext.builder().getOrCreate()\n",
    "\n",
    "sedona = SedonaContext.create(config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c1cb67f",
   "metadata": {},
   "outputs": [],
   "source": [
    "sedona.table(\"wherobots_open_data.overture.transportation_segment\").printSchema()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcf81397",
   "metadata": {},
   "source": [
    "```\n",
    "root\n",
    " |-- id: string (nullable = true)\n",
    " |-- updatetime: timestamp (nullable = true)\n",
    " |-- version: integer (nullable = true)\n",
    " |-- level: integer (nullable = true)\n",
    " |-- subtype: string (nullable = true)\n",
    " |-- connectors: array (nullable = true)\n",
    " |    |-- element: string (containsNull = true)\n",
    " |-- road: string (nullable = true)\n",
    " |-- sources: array (nullable = true)\n",
    " |    |-- element: map (containsNull = true)\n",
    " |    |    |-- key: string\n",
    " |    |    |-- value: string (valueContainsNull = true)\n",
    " |-- bbox: struct (nullable = true)\n",
    " |    |-- minx: double (nullable = true)\n",
    " |    |-- maxx: double (nullable = true)\n",
    " |    |-- miny: double (nullable = true)\n",
    " |    |-- maxy: double (nullable = true)\n",
    " |-- geometry: geometry (nullable = true)\n",
    " |-- geohash: string (nullable = true)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ba53bf5",
   "metadata": {},
   "source": [
    "Learn more about Wherobots open data in the [Wherobots documentation.](https://docs.wherobots.services/1.2.0/tutorials/opendata/introduction/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d54faf71",
   "metadata": {},
   "outputs": [],
   "source": [
    "sedona.table(\"wherobots_open_data.overture.transportation_segment\").createOrReplaceTempView(\"segments\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db6d0df2",
   "metadata": {},
   "outputs": [],
   "source": [
    "segment_df = sedona.sql(\"\"\"\n",
    "    SELECT *\n",
    "    FROM segments\n",
    "    WHERE road LIKE '%isUnderConstruction%'\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50b0a055",
   "metadata": {},
   "outputs": [],
   "source": [
    "segment_df.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29c74be8",
   "metadata": {},
   "source": [
    "```\n",
    "393894\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58eb24c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "SedonaKepler.create_map(df=segment_df, name=\"Under Construction\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17cace9f",
   "metadata": {},
   "source": [
    "![](../img/02-world-view.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "956018b7",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}