Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Openai sdk agent minor fixes #53

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 147 additions & 40 deletions openai_sdk_agent_app_sample_code/01_data_pipeline.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {
Expand All @@ -134,7 +134,16 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "b3bd89b1-83d0-47d4-a3aa-00f32f2494d2",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"### Connect to Databricks (Local IDE only)\n",
"\n",
Expand All @@ -143,8 +152,17 @@
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "87623ad1-a556-4bc7-af9f-aae3f10771f2",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"outputs": [],
"source": [
"from cookbook.databricks_utils import get_cluster_url\n",
Expand Down Expand Up @@ -217,10 +235,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "59b3efc5-0591-4a44-b88d-184003cabfb6",
"showTitle": false,
Expand Down Expand Up @@ -289,8 +310,20 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "bb522de2-004c-44cc-9023-d95e24040bf8",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"outputs": [],
"source": [
"from cookbook.config.data_pipeline.data_pipeline_output import DataPipelineOuputConfig\n",
Expand Down Expand Up @@ -362,10 +395,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "06ee684b-c7bd-4c0e-8fd8-f54416948a5a",
"showTitle": false,
Expand Down Expand Up @@ -395,7 +431,16 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "0f792e9d-e4dc-4dd2-9078-651117dc640c",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"#### 🚫✏️ Write the data pipeline configuration to a YAML\n",
"\n",
Expand All @@ -404,8 +449,20 @@
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "60e5afe7-cd4a-4153-867e-1f81e1c24229",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"outputs": [],
"source": [
"from cookbook.config.data_pipeline import DataPipelineConfig\n",
Expand Down Expand Up @@ -454,6 +511,22 @@
"The code below executes the data pipeline. You can modify the below code as indicated to implement different parsing or chunking strategies or to extract additional metadata fields"
]
},
{
"cell_type": "markdown",
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "f0e36fae-4bb4-48cd-ba83-4574b1d25d43",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"#### 🛑 Make sure to populate the volume with source files before running the parsing code below."
]
},
{
"cell_type": "markdown",
"metadata": {
Expand Down Expand Up @@ -496,10 +569,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "d09fd38c-5b7b-47c5-aa6a-ff571ce2f83b",
"showTitle": false,
Expand Down Expand Up @@ -535,10 +611,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "48a3ab67-2e30-4e39-b05e-3a8ff304fd5b",
"showTitle": false,
Expand Down Expand Up @@ -588,10 +667,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "165706b2-5824-42e7-a22b-3ca0edfd0a77",
"showTitle": false,
Expand Down Expand Up @@ -646,15 +728,36 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"inputWidgets": {},
"nuid": "3556c8be-cd81-4af4-9964-fe3b86b1c16a",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"source": [
"Show any parsing failures or successfully parsed files that resulted in an empty document."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "cbcf821a-19e0-4904-993b-7c89b7af0e49",
"showTitle": false,
"tableResultSettingsMap": {},
"title": ""
}
},
"outputs": [],
"source": [
"\n",
Expand Down Expand Up @@ -710,10 +813,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "02c40228-f933-4af8-9121-ed2efa0985dd",
"showTitle": false,
Expand Down Expand Up @@ -769,10 +875,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "0dfa90f8-c4dc-4485-8fa8-dcd4c7d40618",
"showTitle": false,
Expand Down Expand Up @@ -838,10 +947,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "d53faa42-2a65-40b0-8fc1-6c27e88df6d0",
"showTitle": false,
Expand Down Expand Up @@ -891,10 +1003,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 0,
"metadata": {
"application/vnd.databricks.v1+cell": {
"cellMetadata": {},
"cellMetadata": {
"byteLimit": 2048000,
"rowLimit": 10000
},
"inputWidgets": {},
"nuid": "0cd40431-4cd3-4cc9-b38d-5ab817c40043",
"showTitle": false,
Expand All @@ -915,6 +1030,7 @@
],
"metadata": {
"application/vnd.databricks.v1+notebook": {
"computePreferences": null,
"dashboards": [],
"environmentMetadata": null,
"language": "python",
Expand All @@ -930,16 +1046,7 @@
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.10"
"name": "python"
}
},
"nbformat": 4,
Expand Down
Loading