NeMo-Guardrails.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "23205850",
   "metadata": {},
   "source": [
    "# End-To-End LLM \n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1eb2fea8",
   "metadata": {},
   "source": [
    "## Overview  \n",
    "\n",
    "The End-to-End LLM (Large Language Model) Bootcamp is designed from a real-world perspective that follows the data processing, development, and deployment pipeline paradigm. Attendees walk through the workflow of preprocessing the SQuAD (Stanford Question Answering Dataset) dataset for Question Answering task, training the dataset using BERT (Bidirectional Encoder Representations from Transformers), and executing prompt learning strategy using NVIDIA® NeMo™ and a transformer-based language model, NVIDIA Megatron. Attendees will also learn to optimize an LLM using NVIDIA TensorRT™, an SDK for high-performance deep learning inference, guardrail prompts and responses from the LLM model using NeMo Guardrails, and deploy the AI pipeline using NVIDIA Triton™ Inference Server, an open-source software that standardizes AI model deployment and execution across every workload.\n",
    "\n",
    "\n",
    "### Why End-to-End LLM?\n",
    "\n",
    "Solving real-world problems in the AI domain requires the use of a set of tools (software stacks and frameworks) and the solution process always follows the `data processing`, `development`, and `deployment` pattern. This material is to:\n",
    "- assist AI hackathon participants to learn and apply the knowledge to solve their tasks using NVIDIA software stacks and frameworks\n",
    "- enables bootcamp attendees to solve real-world problem using end-to-end approach (data processing --> development --> deployment)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9ce1706",
   "metadata": {},
   "source": [
    "The End-to-End LLM Bootcamp content contains three labs:\n",
    "- Lab 1: Megatron-GPT\n",
    "- Lab 2: TensorRT-LLM and Triton Deployment with Llama-2-7B Model\n",
    "- Lab 3: NeMo Guardrails\n",
    "\n",
    "The table of contents below will walk you through the `NeMo Guardrails` lab and the activities included will test your understanding of the concept."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b3c7bb1",
   "metadata": {},
   "source": [
    "### Table of Content\n",
    "\n",
    "The following contents will be covered:\n",
    "\n",
    "**Lab 3: NeMo Guardrails**\n",
    "1. [NeMo Guardrails Topical Rails](jupyter_notebook/nemo-guardrails/guardrails/workspace/examples/topical_rail/topical_rail.ipynb)\n",
    "1. [NeMo Guardrails Jailbreak Rails](jupyter_notebook/nemo-guardrails/guardrails/workspace/examples/jailbreak_check/jailbreak_check.ipynb)\n",
    "1. [NeMo Guardrails Grounding Rails](jupyter_notebook/nemo-guardrails/guardrails/workspace/examples/grounding_rail/grounding_rail.ipynb)\n",
    "1. [NeMo Guardrails Moderation Rails](jupyter_notebook/nemo-guardrails/guardrails/workspace/examples/moderation_rail/moderation_rail.ipynb)\n",
    "1. [NeMo Guardrails Langchain and Prompt Templates](jupyter_notebook/nemo-guardrails/guardrails/workspace/examples/custom_prompt_context/custom_prompt_context.ipynb)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dabf1191",
   "metadata": {},
   "source": [
    "### Check your GPU\n",
    "\n",
    "Let's execute the cell below to display information about the CUDA driver and GPUs running on the server by running the nvidia-smi command. To do this, execute the cell block below by giving it focus (clicking on it with your mouse), and hitting `Ctrl-Enter`, or pressing the play button in the toolbar above. If all goes well, you should see some output returned below the grey cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67a54ecf",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "285a3790",
   "metadata": {},
   "source": [
    "### Tutorial Duration\n",
    "\n",
    "The material will be presented 3 labs in a total of 8hrs: 45mins sessions as follows:\n",
    "- NeMo Megatron-GPT Lab: `4hrs: 30mins`\n",
    "- TensorRT-LLM and Triton Deployment with LLama2 7B Model Labs: `1hrs: 10mins`\n",
    "- NeMo Megatron-GPT : `3hrs: 05mins`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e86fea0",
   "metadata": {},
   "source": [
    "### Content Level\n",
    "Beginner to Advanced\n",
    "\n",
    "### Target Audience and Prerequisites\n",
    "The target audience for these labs are researchers, graduate students, and developers who are interested in the End-to-End approach to solving LLM tasks via the use of GPUs. Audiences are expected to have Python programming background Knowledge."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e8d9705",
   "metadata": {},
   "source": [
    "---\n",
    "## Licensing\n",
    "\n",
    "Copyright © 2022 OpenACC-Standard.org. This material is released by OpenACC-Standard.org, in collaboration with NVIDIA Corporation, under the Creative Commons Attribution 4.0 International (CC BY 4.0). These materials include references to hardware and software developed by other entities; all applicable licensing and copyrights apply."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}