Skip to content

Commit

Permalink
add ploomber cloud deployment
Browse files Browse the repository at this point in the history
  • Loading branch information
lfunderburk committed Dec 4, 2023
1 parent 8275fb5 commit b536f3c
Show file tree
Hide file tree
Showing 3 changed files with 285 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,96 @@ jupyter:

## Introduction

Ploomber Cloud is a platform that allows you to deploy Python applications. It provides a CLI to deploy your projects to the cloud and schedule tasks to run at specific times. It also provides a user interface to monitor your tasks and view their logs.
Deploying artificial intelligence and machine learning applications can be a daunting task, especially when it comes to managing the underlying infrastructure. With Ploomber Cloud, you can enjoya platform where AI/ML applications can be deployed with minimal hassle. Here's a comprehensive guide to getting started with Ploomber Cloud and leveraging its deployment capabilities, including its CLI and integration with GitHub for continuous deployment.

## Getting Started with Ploomber Cloud

1. Sign Up
First, head to the [Ploomber Cloud sign-up page](https://platform.ploomber.io/register) and create a free account using your email and a password.

2. Confirm Your Email
After signing up, check your inbox (and spam folder, just in case) for a confirmation email. Click the provided link to activate your account.

3. Sign In
Now, return to Ploomber Cloud and sign in with your credentials.

4. Deploy Your First App
Congratulations, you're all set! Next, explore how to deploy your first application.

## Acquiring an API Key
To interact with Ploomber Cloud, you'll need an [API key](https://docs.cloud.ploomber.io/en/latest/quickstart/apikey.html). After signing up, navigate to your account section and copy the API key.

## Deploying Applications via Command-Line Interface

Install the Ploomber Cloud package using `pip`:

```bash
pip install ploomber-cloud
```

Next, set your API key as an environment variable:

```bash
ploomber-cloud key YOURKEY
```

Initialize a New App

```bash
ploomber-cloud init
```

This will create a `ploomber-cloud.json` file in the directory where you ran the command. This file contains the configuration for your project. You can edit this file to customize your project's configuration. This is what it looks like:

```python
{
"id": "APP_ID",
"type": "APP_TYPE"
}
```

Where your `APP_ID` is a unique identifier for your project and `APP_TYPE` is the type of application you're deploying. Currently, Ploomber Cloud supports the following types of applications:



After initialization, deploy your app using:

```bash
ploomber-cloud deploy
```

The deploy command provides a URL for tracking your deployment's progress.

## Integration with GitHub

Start by storing your API key as a GitHub secret in your repository. This is crucial for GitHub Actions to deploy your projects securely.

Configure GitHub Actions:

Add a YAML file in ``.github/workflows/ploomber-cloud.yaml`` to your repository. This file will contain the workflow configuration.

Sample Workflow Configuration

```yaml
name: Ploomber Cloud

on:
push:
branches:
- main
pull_request:
branches:
- main
schedule:
- cron: "0 0 * * *"

jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Deploy to Ploomber Cloud
uses: ploomber/actions/[email protected]
with:
api-key: ${{ secrets.PLOOMBER_CLOUD_API_KEY }}
```
157 changes: 157 additions & 0 deletions docs/deploying-your-sql-applications/sample-deployment-pc.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,162 @@ jupyter:

# Sample Ploomber Cloud Deployment

In this blog we will explore how we can deploy Python applications with Ploomber Cloud and GitHub actions. We will use a sample project to demonstrate the process. Imagine you need to extract weather data from an API periodically and store it for analysis. You can achieve this by creating a Python script and scheduling its execution with GitHub Actions.

## Initialize data extraction script

This script defines functions to extract weather data, transform it into a DataFrame, and save it as a CSV file. You can replace the latitude and longitude coordinates with locations of your choice.

```python
import requests
import pandas as pd
from dotenv import load_dotenv
import os


def extract_weather_by_lat_lon(api_key, lat, lon):
"""
Extracts weather data from RapidAPI
Parameters
----------
api_key : str
API key for RapidAPI
lat : float
Latitude
lon : float
Longitude
"""
try:
# Perform call
url = "https://weatherapi-com.p.rapidapi.com/forecast.json"

querystring = {"q": f"{lat},{lon}", "days": "5"}

headers = {
"X-RapidAPI-Key": api_key,
"X-RapidAPI-Host": "weatherapi-com.p.rapidapi.com",
}

response = requests.get(url, headers=headers, params=querystring)
response.raise_for_status()
return response.json()
except requests.exceptions.HTTPError as e:
print(e.response.text)
return {}


def transform_json_to_dataframe(response):
"""
Transforms JSON response to dataframe
Parameters
----------
response : dict
Response from API call
"""
try:
unnested_df = pd.json_normalize(
response,
record_path=["forecast", "forecastday", "hour"],
meta=["location", "current"],
)
unnested_df.drop(columns=["location", "current"], inplace=True)

location_df = pd.json_normalize(response["location"])

for col_name in location_df.columns:
unnested_df[col_name] = location_df[col_name][0]

return unnested_df

except KeyError as e:
print("Key Error:", e)
return pd.DataFrame()
except Exception as e:
print("Other Error:", e)
return pd.DataFrame()


def extraction_df_lat_lon(api_key, lat, lon):
"""
Extracts weather data from RapidAPI and transforms it to a dataframe
Parameters
----------
api_key : str
API key for RapidAPI
lat : float
lon : float
Returns
-------
df : pandas.DataFrame
Weather data
"""
response = extract_weather_by_lat_lon(api_key, lat, lon)
return transform_json_to_dataframe(response)

if __name__ == "__main__":
# Load api key
load_dotenv()
api_key = os.getenv("RapidAPI")

# Extract data
latitudes = [
40.7128,
34.0522,
43.6532
]
longitudes = [
-74.0060,
-118.2437,
-79.3832
]
master_list = []
for lat, lon in zip(latitudes, longitudes):
master_list.append(extraction_df_lat_lon(api_key, lat, lon))

# Concatenate all dataframes
df = pd.concat(master_list)

# Save to csv
df.to_csv("weather.csv", index=False)

```

## Visualize the data

Let's visualize the data to see what it looks like. We will use the Plotly package.

```{code-cell} ipython3
import pandas as pd
import plotly.express as px
df = pd.read_csv("weather.csv")
fig = px.scatter_geo(
df[df["time"] > "2023-12-02"],
lat="lat",
lon="lon",
color="region",
hover_name="country",
size="wind_kph",
animation_frame="time",
projection="natural earth",
title="Wind forecast (next 5 days) in kph for cities in the world",
)
fig.show()
```

## Create a GitHub repository and initializing Ploomber Cloud deployment

Create a GitHub repository and add the Python script and Jupyter notebook to it. You can also add a README file to describe your project.

Next, create a Ploomber Cloud account and initialize the deployment. You can do this by running the following command in your terminal:

```bash
ploomber cloud init
```
36 changes: 36 additions & 0 deletions mini-projects/end-to-end/dataextraction.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,18 @@


def extract_weather_by_lat_lon(api_key, lat, lon):
"""
Extracts weather data from RapidAPI
Parameters
----------
api_key : str
API key for RapidAPI
lat : float
Latitude
lon : float
Longitude
"""
try:
# Perform call
url = "https://weatherapi-com.p.rapidapi.com/forecast.json"
Expand All @@ -25,6 +37,14 @@ def extract_weather_by_lat_lon(api_key, lat, lon):


def transform_json_to_dataframe(response):
"""
Transforms JSON response to dataframe
Parameters
----------
response : dict
Response from API call
"""
try:
unnested_df = pd.json_normalize(
response,
Expand All @@ -49,6 +69,22 @@ def transform_json_to_dataframe(response):


def extraction_df_lat_lon(api_key, lat, lon):
"""
Extracts weather data from RapidAPI and transforms it to a dataframe
Parameters
----------
api_key : str
API key for RapidAPI
lat : float
lon : float
Returns
-------
df : pandas.DataFrame
Weather data
"""
response = extract_weather_by_lat_lon(api_key, lat, lon)
return transform_json_to_dataframe(response)

Expand Down

0 comments on commit b536f3c

Please sign in to comment.