Skip to content

Commit

Permalink
Include workshop
Browse files Browse the repository at this point in the history
  • Loading branch information
cfchase committed Oct 31, 2023
1 parent 82e1176 commit 615c934
Show file tree
Hide file tree
Showing 127 changed files with 1,095 additions and 0 deletions.
40 changes: 40 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Publish to GitHub Pages
on:
push:
branches: [main]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
concurrency:
group: pages
cancel-in-progress: true
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
jobs:
build:
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Node.js
uses: actions/setup-node@v3
with:
node-version: '18'
- name: Install Antora
run: npm i antora
- name: Generate Site
run: npx antora workshop/antora-playbook.yml
- name: Setup Pages
uses: actions/configure-pages@v3
- name: Upload artifact
uses: actions/upload-pages-artifact@v2
with:
path: workshop/build/site
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
**/node_modules

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
Expand Down
Empty file added workshop/.gitignore
Empty file.
25 changes: 25 additions & 0 deletions workshop/antora-playbook.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
site:
title: Fraud Detection Workshop
start_page: fraud-detection-workshop::index.adoc
content:
sources:
- url: ./../
start_path: workshop/docs
asciidoc:
attributes:
release-version: master
page-pagination: true
extensions:
- ./lib/tab-block.js
- ./lib/remote-include-processor.js

ui:
bundle:
url: https://github.com/redhat-developer-demos/rhd-tutorial-ui/releases/download/v0.1.9/ui-bundle.zip
snapshot: true
supplemental_files:
- path: ./supplemental-ui
- path: .nojekyll
- path: ui.yml
contents: "static_files: [ .nojekyll ]"

7 changes: 7 additions & 0 deletions workshop/docs/antora.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
name: fraud-detection-workshop
title: Fraud Detection Workshop
version: master
nav:
- modules/ROOT/nav.adoc

start_page: ROOT:index.adoc
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions workshop/docs/modules/ROOT/examples/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/sh
echo "Hello World"
23 changes: 23 additions & 0 deletions workshop/docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
* 1. Projects
** xref:1-01-dashboard.adoc[1. The Dashboard]
** xref:1-02-project-setup.adoc[2. Data Science Projects]
** xref:1-03-data-connections.adoc[3. Storage Data Connections]
*** xref:1-03-01-data-connections-manual.adoc[1. Manual]
*** xref:1-03-02-data-connections-script.adoc[2. Scripted Local]
** xref:1-04-enable-pipelines.adoc[4. Enable Pipelines]
* 2. Workbenches
** xref:2-01-create-workbench.adoc[1. Create a Workbench]
** xref:2-02-jupyter-env.adoc[2. Jupyter]
** xref:2-03-notebooks.adoc[3. Notebooks]
** xref:2-04-experiment-train.adoc[4. Training a Model]
* 3. Model Serving
** xref:3-01-save-model.adoc[1. Saving the Model]
** xref:3-02-deploy-model.adoc[2. Deploy the Model]
** xref:3-03-model-api.adoc[3. Using the API]
// ** xref:3-04-calling-from-application.adoc[4. Calling from an Application]
* 4. Data Science Pipelines
** xref:4-01-gui-pipelines.adoc[1. Workbench Pipeline Editor]
** xref:4-02-sdk-pipelines.adoc[2. Python Pipelines]
31 changes: 31 additions & 0 deletions workshop/docs/modules/ROOT/pages/1-01-dashboard.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
= 1.1 Navigating The Dashboard

include::_attributes.adoc[]

xref:index.adoc[Back to the introduction]

== The OpenShift Data Science Dashboard

include::_attributes.adoc[]

NOTE: We're going to refer to _Red Hat OpenShift Data Science_ as _RHODS_ throughout this workshop.

After logging into the OpenShift Web Console, you should see the OpenShift Data Science Dashboard as a tile in the top right-hand corner. You'll be prompted to log in to the OpenShift Data Science Dashboard using the Developer Sandbox authentication.

image::projects/ocp-console-ds-tile.png[Data Science Dashboard link]

If you are prompted to log in using OpenShift, please go ahead and do so. RHODS will use the same credentials as OpenShift for the dashboard, notebooks, and all other components.

image::projects/login-with-openshift.png[OpenShift Login, 300]

At the OpenShift Data Science Dashboard, you can see the status of any installed and enabled applications as well as *Explore* other available application integrations.

image::projects/dashboard-enabled.png[Dashboard Enabled]

== Navigating to the OCP console

You can navigate back to the OpenShift console in a similar fashion. Simply click on the application switcher to access the OCP console.

image::projects/ds-console-ocp-tile.png[OCP Console Link]

For now, let's stay in the RHODS Dashboard. Now that you are logged into to *Red Hat OpenShift Data Science*, we can move on to creating a project.
42 changes: 42 additions & 0 deletions workshop/docs/modules/ROOT/pages/1-02-project-setup.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
= 1.2 Setting up your Data Science Project

include::_attributes.adoc[]

xref:index.adoc[Back to the introduction]

== The OpenShift Data Science Dashboard

You should be logged into *Red Hat OpenShift Data Science*, and be able to see the dashboard, that looks like this:

image::projects/dashboard-enabled.png[Dashboard Enabled]

You could start a Jupyter notebook from here, but it would be a one-off notebook run in isolation. To do data science as part of a workflow, we need to create a data science project. Projects allow you and your team to organize and collaborate on resources within separated namespaces. From a project you can create multiple workbenches, each with their own Jupyter notebook environment, and each with their own data connections and cluster storage. In addition, they can share models and data with pipelines and model servers.

* Navigate to *Data Science Projects* in the left menu. Here you can see the list of existing projects that you have access to.

image::projects/dashboard-click-projects.png[Data Science Projects List]

From here, you can select an existing project, or create a new one. If you already have an active project you'd like to use, select it now and skip ahead to the xref:1-03-data-connections.adoc[next section]. Otherwise, let's create a new project.


* Click *Create data science project*.

image::projects/create-project-button.png[Create proejct button]

* Enter a user friendly display name and description. It will automatically generate a resource name, but you can change it if you'd like.

image::projects/ds-project-new-form.png[New DS Project Form]

* You can now see its initial state. There are five types of Project resources:

image::projects/ds-project-new.png[New DS Project]

** *Workbenches* are instances of your development and experimentation environment. They typically contain IDEs such as JupyterLab, RStudio and Visual Studio Code.

** A *Cluster storage* is a volume that persists the files and data you're working on within a workbench. A workbench has access to one or more cluster storage instances.

** *Data connections* contain configuration parameters that are required to connect to a data source such as an S3 object bucket.

** *Pipelines* contain the Data Science Pipelines that have been executed within the Project.

** *Models and model servers* allow you to quickly serve a trained model for real-time inference. You can have multiple model servers per Project, and one model server can host multiple models.
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
= 1.3.1 Creating a Data Connection

include::_attributes.adoc[]

xref:index.adoc[Back to the introduction]

NOTE: If you do not have your own s3 compatible storage, or wish to use a disposable local Minio instance, skip this page and use an automated script to *install object storage locally* for the purposes of this workshop: xref:1-03-02-data-connections-script.adoc[Scripted Local Installation]

== Data Connections

Let's create a data connection to your S3 compatible storage bucket. For this you will need all the credential info including the endpoint URL, access key, secret key, region, and bucket name. If you don't have this information, please contact your storage administrator.

* Click *Add data connection* to open the form.

image::projects/ds-project-add-dc.png[Add Data Connection]

* Fill out the form and name your connection `My Storage`. This will be used to save your personal work, including data and models.

image::projects/ds-project-my-storage-form.png[Add My Storage Form]

* Click *Add data connection* to save and close.

Repeat for the pipeline artifacts connection.

NOTE: If you do not intend to complete the pipelines section of the workshop you can skip this step to add the pipeline artifacts connection.

* Click *Add data connection* to open the form.

* Fill out the form and name your connection `Pipeline Artifacts`. This will be where pipeline servers save artifacts. While necessary to run pipelines, we will not be exploring artifacts in this demo.

image::projects/ds-project-pipeline-artifacts-form.png[Add Pipeline Artifacts Form]

* Click *Add data connection* to save and close.

Now you should see your data connections listed in the project.

image::projects/ds-project-dc-list.png[]
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
= 1.3.2 Scripted Data Connection Installation

include::_attributes.adoc[]

xref:index.adoc[Back to the introduction]


NOTE: This script will install two data connections and two Minio buckets as s3 compatible storage into your data science project for convenience. These are *not* meant for production usage. It is based on the instructions for installing Minio in this https://ai-on-openshift.io/tools-and-applications/minio/minio/[guide]. It will create a random user and password for security. If you wish to connect to your own storage, please refer to these instructions: xref:1-03-01-data-connections-manual.adoc[Manually Adding Data Connections]

== Finding the OpenShift Project Name

We're going to want to update the project we created in the previous section. To do that, we need to navigate to the project in the OpenShift Console. First, we must take note of the project resource name when we created it.

image::projects/ds-project-new-form-resource.png[New Project Form Resource Name]

You can also find the resource name on your list of projects by hovering over the *?* icon next to the project name.

image::projects/ds-project-list-resource-hover.png[Project List Resource Name]


== Navigate to the OpenShift Project

NOTE: This section details how to start an OpenShift job to install the s3 compatible storage and associated data connection secrets. If you are knowledgeable in OpenShift and can access the cluster from the command line, you can simply execute `oc apply -n <your-project-name/> -f https://github.com/cfchase/fraud-detection-notebooks/raw/main/setup/setup-s3.yaml`.

Now that we have the resource name, we can navigate to the project in the OpenShift Console. Click on the *OpenShift Console* link in the application switcher top right corner of the OpenShift Web Console.

image::projects/ds-project-ocp-link.png[OpenShift Console Link]

* Click the *+* icon in the navbar to create new resources.

image::projects/ocp-console-add-icon.png[Add Resources Icon]

* Select your project from the list of projects and verify the correct project is now listed.

image::projects/ocp-console-select-project.png[Select Project]

image::projects/ocp-console-project-selected.png[Project Selected]

[.lines_space]
[.console-input]
[source, yaml]
----
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: demo-setup
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: demo-setup-edit
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: edit
subjects:
- kind: ServiceAccount
name: demo-setup
---
apiVersion: batch/v1
kind: Job
metadata:
name: create-s3-storage
spec:
selector: {}
template:
spec:
containers:
- args:
- -ec
- |-
echo -n 'Setting up Minio instance and data connections'
oc apply -f https://github.com/cfchase/fraud-detection-notebooks/raw/main/setup/setup-s3-no-sa.yaml
command:
- /bin/bash
image: image-registry.openshift-image-registry.svc:5000/openshift/tools:latest
imagePullPolicy: IfNotPresent
name: create-s3-storage
restartPolicy: Never
serviceAccount: demo-setup
serviceAccountName: demo-setup
----


== Data Connections

We will be using S3 compatible object storage, such as Ceph, Minio, or AWS S3, to store the data and models used in this workshop. To access the data and models, we will use a Data Connection, which is a resource that contains the configuration parameters required to connect to an object storage bucket.

We will need 2 buckets.

1. *My Storage* - Personal use for models and data. We can reuse that bucket and connection for notebooks and model servers.
2. *Pipelines Artifacts* - This is used as storage for your pipeline artifacts and required when creating a pipeline server. We'll keep it separate from the first bucket for clarity.

You can choose to use your own object storage and learn how to configure it manually
22 changes: 22 additions & 0 deletions workshop/docs/modules/ROOT/pages/1-03-data-connections.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
= 1.3 Object Storage with Data Connections

include::_attributes.adoc[]

xref:index.adoc[Back to the introduction]

== Data Connections

We will be using S3 compatible object storage, such as Ceph, Minio, or AWS S3, to store the data and models used in this workshop. To access the data and models, we will use a Data Connection, which is a resource that contains the configuration parameters required to connect to an object storage bucket.

We will need 2 buckets.

1. *My Storage* - Personal use for models and data. We can reuse that bucket and connection for notebooks and model servers.
2. *Pipelines Artifacts* - This is used as storage for your pipeline artifacts and required when creating a pipeline server. We'll keep it separate from the first bucket for clarity.

You can choose to:

* Create connections to your own S3 compatible object storage. Click here for instructions on how to do that: xref:1-03-01-data-connections-manual.adoc[Manually Adding Data Connections]

OR

* Use an automated script to install object storage locally for the purposes of this workshop: xref:1-03-02-data-connections-script.adoc[Scripted installation of local Minio instance]
28 changes: 28 additions & 0 deletions workshop/docs/modules/ROOT/pages/1-04-enable-pipelines.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
= 1.4 Enable Data Science Pipelines

include::_attributes.adoc[]

xref:index.adoc[Back to the introduction]

This section will prepare your workspace for Section $ xref:4-01-gui-pipelines.adoc[Data Science Pipelines]. You can do this now to save time later, but you can also skip and return to this section.

NOTE: If you do not intend to complete the pipelines section of the workshop you can skip this step and move on to the next section, xref:2-01-create-workbench.adoc[Create a Workbench]

== Create a Pipeline Server

* Click *Create pipeline server*

image::projects/ds-project-create-pipeline-server.png[Create pipeline server button]

* Select the `Pipeline Artifacts` data connection.

image::projects/ds-project-create-pipeline-server-form.png[Selecting the Pipeline Artifacts data connection]

* Leave the database configuration as the default.

* Click *Configure*.

Pipelines should now be enabled for your project and the *Create pipeline server* button should disappear.

image::projects/ds-project-create-pipeline-server-complete.png[Create pipeline server complete]

Loading

0 comments on commit 615c934

Please sign in to comment.