Skip to content

Commit

Permalink
Merge pull request #56 from reyg3/fire-weather
Browse files Browse the repository at this point in the history
Added new projects and sections to reflect most recent works.
  • Loading branch information
reyg3 authored Jan 5, 2024
2 parents b1ef43f + b184453 commit 32f9d5a
Show file tree
Hide file tree
Showing 5 changed files with 157 additions and 24 deletions.
83 changes: 60 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,36 @@

<br />

[Collaborators](#research-collaborators) | [Proposals](#proposals) | [Projects](#projects) | [Publications](#publications) | [Students](#student-contributors)
[Collaborators](#research-collaborators) | [Proposals](#proposals) | [Technology Transfer](#technology-transfer) | [Projects](#projects) | [Publications](#publications) | [Students](#student-contributors)

<br />

# <b>Overview</b>
# AIR Technical Research Applications Development

## Who we are
## Overview

### Who we are
We are the Technical Research Applications Development group found within the [Arizona Institute for Resilience (AIR)](https://air.arizona.edu). We are a group of technology professionals and students who provide a variety of technical services and collaborations to the [Centers and Programs](https://air.arizona.edu/centers-programs) found within AIR. Our project collaborations span the globe and make meaningful impacts from the desert southwestern United States to the Middle East and places in between. AIR explores and develops solutions with campus and community partners that serve human and natural communities on a global scale by engaging a full array of disciplines, professional schools, international capacity, and entrepreneurial opportunities.

The research applications group is led by [Rey Granillo](https://github.com/reyg3) Director of Technology and Research Computing, [Leland Boeman](https://github.com/lboeman) Research & Development Software Engineer, and [Thomas Weiss](https://github.com/tweissaz) Research & Development Systems Engineer.

<br />

## How we do it
### Development Process
### Our proficiencies
- Research focused software development
- Cloud computing
- Data assimilation, processing, and presentation
- Database architecture and development
- Application Programming Interface (API) development
- Remote sensing
- LoRaWAN network gateways
- Designing and building microcontrollers
- 3D printing
- Machine learning

### How we do it
#### Development Process
By developing and combining technologies, we incorporate novel technical solutions into new and existing research. Using a collaborative model, versus the typical service model found most in Information Technology, we can better understand project data, the research questions being asked, and provide higher quality research collaborations. Having that greater understanding has led us to find correlations in data that were not previously realized which have led to new research project ideas and proposals. We collaborate with researchers during the proposal process and can provide a proof-of-concept application or tool showing what is technically feasible if project funding is awarded. Upon project completion, we aid with the technical writing portions of publications which outline our findings and technical solutions that were implemented.

### Student Support
#### Student Support
We rely on student support and participation across all research projects. Student participation ranges from software development, technical hardware implementation, database architecture design and implementation, ML/AI research, and publication writing. To support these efforts, AIR has established a student research computing working group that primarily meets during the Fall and Spring semesters. This working group is a forum where we discuss new research technologies, ask questions, and where students can report on their current research activities. The intention is to create research focussed critical thinking processes that can generate new ideas and concepts across various projects and collaborations.

Our current student cohort includes:
Expand All @@ -35,7 +47,7 @@ Our current student cohort includes:
- [Lily McMullen](https://github.com/lilymcmullen)
- Majors: Natural Resources (Conservation Biology) and Computer Science

### Research Collaborators
## Research Collaborators
In addition to our core research applications development team, we've had a number of collaborators across AIR programs and affiliated groups across the University of Arizona.

<table>
Expand Down Expand Up @@ -67,7 +79,7 @@ In addition to our core research applications development team, we've had a numb
Eller Doctoral Student Economics</td>
<td><a href="https://anthropology.arizona.edu/people/rachel-rosenbaum"><img src="images/headshots/RosenbaumRachel.jpeg" width="25"> Rachel Rosenbaum </a>
<br />
PhD Candidate School of Anthropology</td>
School of Anthropology</td>
<td><a href="https://air.arizona.edu/person/nancy-petersen"><img src="https://air.arizona.edu/sites/default/files/2022-12/NancyP.jpg" width="25"> Nancy Petersen </a>
<br />
AIR Haury Program</td>
Expand Down Expand Up @@ -165,7 +177,32 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,
<br />

# <b>Projects</b>
## Technology Transfer

>### **National Science Foundation Innovation Corps (NSF I-Corps) Training Program**
> The U.S. National Science Foundation’s Innovation Corps (I-Corps™) program is an immersive, entrepreneurial training program that facilitates the transformation of invention to impact. This immersive, seven-week experiential training program prepares scientists and engineers to extend their focus beyond the university laboratory — accelerating the economic and societal benefits of NSF-funded and other basic research projects that are ready to move toward commercialization. Visit the [NSF's I-Corps website](https://new.nsf.gov/funding/initiatives/i-corps) for more information.
>#### **Desert and Pacific Regional Hub 2023 - Cohort 6 - Fire Data Dynamics**
>AIR's technology team is working with Tech Launch Arizona on licensing for our Fire Data Dynamics prototype to help make meaningful impacts in fire prediction, response, and analysis. As part of this licensing process, we participated in NSF's I-Corps where we learned about the Lean Startup Methodology with a strong focus on customer discovery and learning about the challenges being faced to ensure we are providing a solution with meaningful impact.
<br />

## Projects

>### **Fire Data Dynamics**
>With data from the National Interagency Fire Center's (NIFC) Southwest Coordination Center (SWCC), and with funding from the University of Arizona's Technology and Research Initiative Fund (TRIF), we developed a fire data dashboard prototype designed to be used by and inform decision makers during fire events. Using data from past fire events in Arizona and New Mexico, our dashboard displays fire occurrences and their locations, fire burn perimeters (when available), and relevant data from Remote Automated Weather Stations (RAWS) during those fire events. The current prototype also allows stakeholders to export this historical data for use with their own analysis. In addition, new features are being developed to incorporate a live fire viewer, perform predictive machine learning analysis for determining fire weather and occurrence likelihood, and incorporating Weather Research & Forecasting Model (WRF) to inform decision makers of the impact weather could have during fire events.
>
![Fire Dashboard image](https://github.com/uaenvironment/uaenvironment.github.io/blob/master/images/firedatadashboard.png?raw=true)

>### **Remote Automated Weather Stations (RAWS) Machine learning**
>Leveraging historical RAWS data from the past 20 years alongside popular machine learning technologies, we were able to build models to predict wind speed, temperature, and relative humidity conditions for 6-, 12-, and 24-hour time periods.
>#### RAWS Data
>20 years of weather data was pulled for specific stations in AZ during this prototype phase. The data was then cleaned and scaled for use with the intended machine learning technologies.
>#### Python Package
>A custom RAWSTraining Python package was developed to easily clean and prepare data for use with Tensorflow and Keras to build multi-step forecasting models. With custom classes used for Training and making Predictions, training and testing took only a couple of hours for a single station to be deployed (with non-GPU enabled hardware).
>#### Web Application
>An application built with Vue3, Firebase, and Firestore was launched to show the predictions and their respective stations. This application featured flagging for potential fireweather conditions, and also assessed the models in realtime to showcase the validity of the predictions being made.
>
>Future implementations of this project will exist within the Fire Data Dynamics prototype.
>### **Carbon-Econ Plotter**
>Methane emissions have an outsized impact on climate, and are increasingly receiving attention from policymakers and the scientific community. We map methane plume emission rates measured by [Carbon Mapper](https://carbonmapper.org) onto tract-level demographics from the U.S. Census Bureau to explore environmental justice issues associated with the methane emissions. Carbon Mapper collects methane emission data through their airborne pilot projects with advanced remote sensing technology. Census tract-level demographics are obtained from the 2009-2012 five-year moving average [American Community Survey](https://www.census.gov/programs-surveys/acs). This map and scatter plots can be viewed at [Carbon Plotter](https://carbon-plotter.air.arizona.edu/map) website.
Expand All @@ -182,10 +219,10 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,
<br />


>## **Monsoon Game Repositories**
>### **Monsoon Game Repositories**
>The following projects all pertain to various aspects of the AIR [Monsoon Fantasy Game](https://monsoonfantasy.arizona.edu). These repositories include the detailed scoring method and a post game analysis.
>
>>### **Monsoon Fantasy Game**
>>#### **Monsoon Fantasy Game**
>>In Monsoon Fantasy, players estimate the total monthly precipitation at each of the five major cities in the U.S. Southwest Monsoon region: Tucson, Phoenix, Flagstaff, Albuquerque, and El Paso. Points are awarded each month depending on the accuracy of the estimate compared to the actual observed rainfall. The goal is to accumulate the most points over the July, August, September period.
>>
>| | |
Expand All @@ -195,7 +232,7 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,
>
><br />
>
>>### **Monsoon Game Scoring Method**
>>#### **Monsoon Game Scoring Method**
>><b>Contributors:</b>
>>
>><a href="https://github.com/uaenvironment/monsoon-game-scoring-method/graphs/contributors">
Expand All @@ -210,7 +247,7 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,
>
><br />
>
>>### **Monsoon Post Game Analysis**
>>#### **Monsoon Post Game Analysis**
>>When signing up to play Monsoon Fantasy, players had the option to fill out profile questions. These questions asked things such as
>>- How many monsoon seasons have you experienced while living in the southwest?
>>- How would you rate your understanding of the monsoon system?
Expand All @@ -226,13 +263,13 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,

<br />

>## **Monsoon Data Collection and Processing**
>### **Monsoon Data Collection and Processing**
>
>The following projects are geared towards improving the availability of monsoon related meteorological data in Arizona by providing a centralized and persistent source for otherwise ephemeral observation data and demonstrating the value of that data through visualization and machine learning applications.
>
><br />
>
>>### **Monsoon Scraper**
>>#### **Monsoon Scraper**
>>This project centralizes public data from several different Flood Control District (FCD) networks across the state of Arizona. This data is stored in a cloud based data warehouse and serves as the central data source for a number of monsoon related projects and research. To gather this data we have written a number of applications that run at different intervals dependent on the different FCD network implementations. These applications run on 15 minute to 1 hour intervals. These time intervals are required in order to obtain incremental precipitation data readings which are not available if gathering data on an hour or day interval. In addition to precipitation data, some FCD sensors also report temperature, pressure, humidity, and stream flow intensity in washes.
>>
>>This dataset consists of the following remote sensing precipitation networks along with their API programmatic names. Additional networks will be added as they are implemented.
Expand All @@ -249,21 +286,21 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,
>
><br />
>
>>### **Monsoon Data Application Programming Interface (API)**
>>#### **Monsoon Data Application Programming Interface (API)**
>>Once data from the Monsoon Scraper project was gathered, we developed a REST API to programmatically query the dataset. The API contains a number of custom routes designed to query specific sets of data. Some of these routes include our monsoon route which returns precipitation totals from specified networks or sensors between June 15th - September 30th of provided years, a flood route which returns data from flood gauge sensors typically found in washes, a sensors metadata route which returns metadata of specific sensors, and a readings route which queries specific sensors or networks using a provided date range.
>>
>>Currently, API keys are only issued to researchers working with this dataset. There are plans to expand this audience in the future.
>
><br />
>
>>### **Monsoon Plotter**
>>#### **Monsoon Plotter**
>>Monsoon Plotter is used to visually represent the data gathered via the Monsoon Scraper project which collects data from the state of Arizona flood control district (FCD) remote sensing networks. There are a handful of networks that can be plotted and more will be added as we expand our Monsoon Scraper project to gather more data. There is also a limited CSV export feature available of the specific data points chosen to be plotted. For full exports of data an API key is required to make programmatic calls to the Monsoon API.
>>
>![Monsoon Plotter](https://github.com/uaenvironment/uaenvironment.github.io/blob/master/images/monsoon_plotter.png?raw=true)
>
><br />
>
>>### **Monsoon API Package/CLI Tool**
>>#### **Monsoon API Package/CLI Tool**
>>This Python package serves as a wrapper to simplify REST API calls to the monsoon scraper data warehouse. This is the same dataset that is visually represented in our monsoon plotter found at [monsoon.environment.arizona.edu](https://monsoon.environment.arizona.edu). The plotter allows a limited export of the data dependent on the date range and sensor network being plotted. This package allows you to incorporate our monsoon dataset into a local codebase for processing.
>>
>>This package also contains a Command Line Interface (CLI) tool for those who prefer to work within a CLI instead of the Python package.
Expand All @@ -272,7 +309,7 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,
>
><br />
>
>>### **Monsoon Machine Learning**
>>#### **Monsoon Machine Learning**
>>This repository contains code in R and Python that demonstrates how to create basic machine learning algorithms. It then takes historical weather data from the Tucson International Airport, precipitation data from our Monsoon API, and storm data from NOAA and applies these machine learning algorithms in an attempt to accurately predict flooding using historic data and a database of notable flood and rainfall events.
>>
>![Monsoon Machine Learning](https://github.com/uaenvironment/uaenvironment.github.io/blob/master/images/gradientBoosting.png?raw=true)
Expand Down Expand Up @@ -313,7 +350,7 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,

<br />

# Publications
## Publications
>>Dharma H, Granillo R.L. III, Boeman L, McMahan B, Crimmins MA (2023) [Data Aggregation, ML ready Datasets, and an API: Leveraging diverse data to create enhanced characterizations of monsoon flood risk. Frontiers in Climate](https://www.frontiersin.org/articles/10.3389/fclim.2023.1107363/full). Front. Clim. doi: 10.3389/fclim.2023.1107363
>
>>Guido, Z., McMahan, B., Hoy, D., Larsen, C., Delgado, B., Granillo, R. L., III, & Crimmins, M. (2022). Public Engagement on Weather and Climate with a Monsoon Fantasy Forecasting Game, Bulletin of the American Meteorological Society. <https://journals.ametsoc.org/view/journals/bams/aop/BAMS-D-22-0003.1/BAMS-D-22-0003.1.xml>
Expand All @@ -330,7 +367,7 @@ managers, and EXPLORE opportunities for aggregation, management, visualization,
<br />

# Student Contributors
## Student Contributors

### Current:

Expand Down
Binary file added images/firedatadashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/fwmachinelearning-graph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/fwmachinelearning-map.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 32f9d5a

Please sign in to comment.