From 07d143ea799b746b692f194cc4355e76ff7a7e38 Mon Sep 17 00:00:00 2001 From: Adam Gardner Date: Wed, 23 Oct 2024 13:11:55 +1000 Subject: [PATCH] Deployed 957443d with MkDocs version: 1.6.0 --- how-it-works-rag/index.html | 67 +++----------------------------- how-it-works-standard/index.html | 65 +++---------------------------- index.html | 24 +++--------- prerequisites/index.html | 24 +++--------- resources/index.html | 2 +- search/search_index.json | 2 +- setup/index.html | 24 +++--------- startup/index.html | 24 +++--------- use-demo/index.html | 24 +++--------- visualise-dt/index.html | 24 +++--------- whats-next/index.html | 62 +++-------------------------- 11 files changed, 48 insertions(+), 294 deletions(-) diff --git a/how-it-works-rag/index.html b/how-it-works-rag/index.html index 52b4ec3..05b67e3 100755 --- a/how-it-works-rag/index.html +++ b/how-it-works-rag/index.html @@ -71,11 +71,6 @@
- - - Skip to content - -
@@ -308,17 +303,6 @@ - - @@ -329,32 +313,6 @@ - - - - @@ -537,23 +495,6 @@ - - -
@@ -584,9 +525,13 @@

3. How it works (RAG)

  • It contacts Ollama to produce an embedding of the user input
  • With the embedding, reach out to PineCone to find documents relevant to the user input
  • Use the documents to perform prompt engineering and send it to Ollama to produce the travel recommendation
    • -
  • Prosess the answer received
    • +
  • Process the answer received
    • + +
    + -

    >> Click here to continue with the exercise#

    +
    diff --git a/how-it-works-standard/index.html b/how-it-works-standard/index.html index da1d04b..75f8dea 100755 --- a/how-it-works-standard/index.html +++ b/how-it-works-standard/index.html @@ -71,11 +71,6 @@
    - - - Skip to content - -
    @@ -288,17 +283,6 @@ - - @@ -309,32 +293,6 @@ - - - - @@ -537,23 +495,6 @@ - - -
    @@ -582,7 +523,11 @@

    2. How it works (Standard)

  • The response is returned and cached so that subsequent calls for the same destination (eg. Sydney) are served from the cache. This saves roundtrips to ChatGPT and thus $.
    • -

    >> Click here to continue with the exercise#

    +
    + +
    diff --git a/index.html b/index.html index aafff3d..132584c 100755 --- a/index.html +++ b/index.html @@ -329,15 +329,6 @@ - - -
  • - - - >> Click here to start the tutorial... - - -
    • @@ -599,15 +590,6 @@ - - -
  • - - - >> Click here to start the tutorial... - - -
    • @@ -672,7 +654,11 @@

    Compatibility -

    >> Click here to start the tutorial...#

    +
    + +
    diff --git a/prerequisites/index.html b/prerequisites/index.html index 597464e..725a9bf 100755 --- a/prerequisites/index.html +++ b/prerequisites/index.html @@ -369,15 +369,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -559,15 +550,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -599,7 +581,11 @@

    4. Demo Prerequisites

    Why is a Paid OpenAI Account Required?#

    OpenAI / ChatGPT severely limits the ability for API access if you do not have credit. Adding a small amount of credit ($2-$3) is the best way to make this (and all other ChatGPT demos) run smoothly.

    This demo uses gpt 4o mini. We have developed, tested and demoed this repository hundreds of times and still have money left from the initial $5 credit load.

    -

    >> Click here to continue with the exercise#

    +
    + +
    diff --git a/resources/index.html b/resources/index.html index 7c4ca6f..31f10bd 100755 --- a/resources/index.html +++ b/resources/index.html @@ -522,7 +522,7 @@

    Resources#

    diff --git a/search/search_index.json b/search/search_index.json index c5a24ca..85b28d2 100755 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Large Language Model Observability with Dynatrace","text":"

    Support Policy

    This is a demo project created by the Developer Relations team at Dynatrace, showcasing integrations with open source technologies.

    Support is provided via GitHub issues only. The materials provided in this repository are offered \"as-is\" without any warranties, express or implied. Use them at your own risk.

    View the Code

    The code for this repository is hosted on GitHub. Click the \"View Code on GitHub\" link above.

    Demo application for giving travel advice written in Python. Observability signals by OpenTelemetry and OpenLLMetry.

    The demo application has two different versions:

    The standard version uses OpenAI ChatGPT to generate advice for a given destination and use Weaviate as a cache layer. The RAG version uses Ollama to generate advice for a given destination and PineCone to fetch additional context to refine the advice provided by Ollama.

    This hands-on is also available as an on-demand webinar.

    "},{"location":"#standard-version-architecture","title":"Standard Version Architecture","text":""},{"location":"#rag-version-architecture","title":"RAG Version Architecture","text":""},{"location":"#compatibility","title":"Compatibility","text":"Deployment Tutorial Compatible Dynatrace Managed \u2714\ufe0f Dynatrace SaaS \u2714\ufe0f"},{"location":"#click-here-to-start-the-tutorial","title":">> Click here to start the tutorial...","text":""},{"location":"cleanup/","title":"Cleanup","text":"

    Go to https://github.com/codespaces and delete the codespace which will delete the demo environment.

    "},{"location":"how-it-works-rag/","title":"3. How it works (RAG)","text":"

    The \"Retrieval-Augmented Generation\" (RAG) version (available on the ollama-pinecone branch) mimicks training an LLM on an internal knowledgebase. It will produce custom destination advice for places the system has explicitly been trained on (the files in the destinations folder). Namely, Bali and Sydney. For other locations, the model will provide an answer based on its own knowledge. It is based on Ollama and uses PineCone as a Vector database. The RAG pipeline is built using LangChain.

    The RAG version of the demo mimicks training an LLM on an internal knowledgebase.

    When the application starts, files inside the destinations folder are read, processed, and stored in PineCone for later lookup. Afterwards, each request goes through the LangChain RAG pipeline, which performs the following steps:

    "},{"location":"how-it-works-rag/#click-here-to-continue-with-the-exercise","title":">> Click here to continue with the exercise","text":""},{"location":"how-it-works-standard/","title":"2. How it works (Standard)","text":"

    The user interacts with the demo app (travel advisor) on port 30100. The app is monitored either via native OpenTelemetry.

    The user enters a destination (eg. Sydney):

    "},{"location":"how-it-works-standard/#click-here-to-continue-with-the-exercise","title":">> Click here to continue with the exercise","text":""},{"location":"prerequisites/","title":"4. Demo Prerequisites","text":"

    To run this demo you will need:

    "},{"location":"prerequisites/#why-is-a-paid-openai-account-required","title":"Why is a Paid OpenAI Account Required?","text":"

    OpenAI / ChatGPT severely limits the ability for API access if you do not have credit. Adding a small amount of credit ($2-$3) is the best way to make this (and all other ChatGPT demos) run smoothly.

    This demo uses gpt 4o mini. We have developed, tested and demoed this repository hundreds of times and still have money left from the initial $5 credit load.

    "},{"location":"prerequisites/#click-here-to-continue-with-the-exercise","title":">> Click here to continue with the exercise","text":""},{"location":"resources/","title":"Resources","text":""},{"location":"setup/","title":"5. Setup","text":""},{"location":"setup/#create-openai-api-token","title":"Create OpenAI API Token","text":"

    Go to https://platform.openai.com/api-keys and create a new API Key.

    "},{"location":"setup/#format-dynatrace-url","title":"Format Dynatrace URL","text":"

    Make a note of your Dynatrace URL, it should be in the following format:

    https://ENVIRONMENT-ID.live.dynatrace.com\n

    For example:

    https://abc12345.live.dynatrace.com\n
    "},{"location":"setup/#create-dynatrace-token","title":"Create Dynatrace Token","text":"

    In Dynatrace, press Ctrl + k and search for access tokens. Choose the first option.

    "},{"location":"setup/#dt_api_token","title":"DT_API_TOKEN","text":"

    Create an API token with these permissions:

    This token will be used by the OpenTelemetry collector and k6 to send data to Dynatrace. The setup script which runs automatically when the codespace is created also uses this to configure span attribute capture rules in Dynatrace this means the relevant OpenTelemetry span attributes will automatically be stored.

    "},{"location":"setup/#recap","title":"\ud83d\udd01 Recap","text":"

    You should now have 3 pieces of information:

    When you have these pieces of information, you can proceed.

    "},{"location":"setup/#click-here-to-continue-with-the-exercise","title":">> Click here to continue with the exercise","text":""},{"location":"startup/","title":"6. Start The Demo","text":""},{"location":"startup/#time-to-fire-it-up","title":"\ud83c\udd99 Time to Fire it up","text":"

    Choose one of the following options to start the codespace:

    "},{"location":"startup/#launch-standard-version","title":"Launch Standard Version","text":""},{"location":"startup/#launch-rag-version","title":"Launch RAG Version","text":"

    Leave the top section blank and provide your values in the Recommended secrets form.

    After the codespaces has started (in a new browser tab), the post creation script should begin. This will install everything and will take a few moments.

    When the script has completed, a success message will briefly be displayed (it is so quick you'll probably miss it) and an empty terminal window will be shown.

    TODO: Needs new image

    You may now proceed.

    "},{"location":"startup/#click-here-to-continue-with-the-exercise","title":">> Click here to continue with the exercise","text":""},{"location":"use-demo/","title":"7. Use the Demo","text":""},{"location":"use-demo/#accessing-and-using-demo","title":"Accessing and Using Demo","text":"

    In the codespace, switch to the Ports tab. Right click port 30100 and choose Open in Browser

    A new browser tab will open and you should see the demo.

    "},{"location":"use-demo/#using-llm-based-destination-search","title":"Using LLM-based Destination Search","text":"

    Type the name of a destination (eg. Vienna) into the search bar and click the Advise button.

    "},{"location":"use-demo/#what-happens-next","title":"What Happens Next?","text":"

    If you search for Vienna again, this time, the re1sult will be served from the cache - saving you the roundtrip (time and $) to OpenAI / ChatGPT.

    "},{"location":"use-demo/#customer-feedback","title":"Customer Feedback","text":"

    Click the \ud83d\udc4d and / or \ud83d\udc4e buttons to indicate your satisfaction level of the result.

    Clicking these icons will log a message. This log line is then retrieved and processed using DQL in the \"User Sentiment Analysis\" section of the dashboard.

    "},{"location":"use-demo/#click-here-to-continue-with-the-exercise","title":">> Click here to continue with the exercise","text":""},{"location":"visualise-dt/","title":"Visualising Data in Dynatrace","text":""},{"location":"visualise-dt/#uploading-the-dashboards","title":"Uploading the Dashboards","text":"

    This demo comes with several prebuilt dashboards. Do the following in Dynatrace.

    Repeat this process for all the dashboards inside dynatrace/dashboards/*

    "},{"location":"visualise-dt/#distributed-traces","title":"Distributed Traces","text":"

    The application emits distributed traces which can be viewed in Dynatrace:

    Remember that only the very first requests for a given destination will go out to OpenAI. So expect many many more cached traces than \"live\" traces.

    "},{"location":"visualise-dt/#trace-with-openai","title":"Trace with OpenAI","text":"

    A \"full\" call to OpenAI looks like this. Notice the long call halfway through the trace to openai.chat. These traces take much longer (3 seconds vs. 500ms).

    "},{"location":"visualise-dt/#trace-to-weaviate-cache","title":"Trace to Weaviate Cache","text":"

    A call which instead only hits the on-cluster Weaviate cache looks like this.

    Notice that it is much quicker.

    The response TTL (max time that a cached prompt is considered \"fresh\") is checked and if the response is \"still fresh\" (ie. TTL < stale time) the cached value is returned.

    Notice the cached prompt is 123s. The max age (TTL is (by default) 60 minutes. Therefore the prompt is not outdated and thus returned to the user as valid.

    "},{"location":"visualise-dt/#demo-complete","title":"\ud83c\udf89 Demo Complete","text":"

    The demo is now complete. Continue to cleanup your environment.

    "},{"location":"visualise-dt/#cleanup-resources-to-avoid-github-charges","title":">> Cleanup resources to avoid GitHub charges","text":""},{"location":"whats-next/","title":"What's Next?","text":"

    Head to the resources page to see other LLM Observability content.

    "},{"location":"whats-next/#resources","title":"Resources","text":""},{"location":"snippets/disclaimer/","title":"Disclaimer","text":"

    Support Policy

    This is a demo project created by the Developer Relations team at Dynatrace, showcasing integrations with open source technologies.

    Support is provided via GitHub issues only. The materials provided in this repository are offered \"as-is\" without any warranties, express or implied. Use them at your own risk.

    "},{"location":"snippets/preview-functionality/","title":"Preview functionality","text":"

    Preview Functionality

    This demo uses functionality that is still in Preview.

    After creating a Dynatrace environment, please sign up for this preview functionality.

    "},{"location":"snippets/view-code/","title":"View code","text":"

    View the Code

    The code for this repository is hosted on GitHub. Click the \"View Code on GitHub\" link above.

    "}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Large Language Model Observability with Dynatrace","text":"

    Support Policy

    This is a demo project created by the Developer Relations team at Dynatrace, showcasing integrations with open source technologies.

    Support is provided via GitHub issues only. The materials provided in this repository are offered \"as-is\" without any warranties, express or implied. Use them at your own risk.

    View the Code

    The code for this repository is hosted on GitHub. Click the \"View Code on GitHub\" link above.

    Demo application for giving travel advice written in Python. Observability signals by OpenTelemetry and OpenLLMetry.

    The demo application has two different versions:

    The standard version uses OpenAI ChatGPT to generate advice for a given destination and use Weaviate as a cache layer. The RAG version uses Ollama to generate advice for a given destination and PineCone to fetch additional context to refine the advice provided by Ollama.

    This hands-on is also available as an on-demand webinar.

    "},{"location":"#standard-version-architecture","title":"Standard Version Architecture","text":""},{"location":"#rag-version-architecture","title":"RAG Version Architecture","text":""},{"location":"#compatibility","title":"Compatibility","text":"Deployment Tutorial Compatible Dynatrace Managed \u2714\ufe0f Dynatrace SaaS \u2714\ufe0f "},{"location":"cleanup/","title":"Cleanup","text":"

    Go to https://github.com/codespaces and delete the codespace which will delete the demo environment.

    "},{"location":"how-it-works-rag/","title":"3. How it works (RAG)","text":"

    The \"Retrieval-Augmented Generation\" (RAG) version (available on the ollama-pinecone branch) mimicks training an LLM on an internal knowledgebase. It will produce custom destination advice for places the system has explicitly been trained on (the files in the destinations folder). Namely, Bali and Sydney. For other locations, the model will provide an answer based on its own knowledge. It is based on Ollama and uses PineCone as a Vector database. The RAG pipeline is built using LangChain.

    The RAG version of the demo mimicks training an LLM on an internal knowledgebase.

    When the application starts, files inside the destinations folder are read, processed, and stored in PineCone for later lookup. Afterwards, each request goes through the LangChain RAG pipeline, which performs the following steps:

    "},{"location":"how-it-works-standard/","title":"2. How it works (Standard)","text":"

    The user interacts with the demo app (travel advisor) on port 30100. The app is monitored either via native OpenTelemetry.

    The user enters a destination (eg. Sydney):

    "},{"location":"prerequisites/","title":"4. Demo Prerequisites","text":"

    To run this demo you will need:

    "},{"location":"prerequisites/#why-is-a-paid-openai-account-required","title":"Why is a Paid OpenAI Account Required?","text":"

    OpenAI / ChatGPT severely limits the ability for API access if you do not have credit. Adding a small amount of credit ($2-$3) is the best way to make this (and all other ChatGPT demos) run smoothly.

    This demo uses gpt 4o mini. We have developed, tested and demoed this repository hundreds of times and still have money left from the initial $5 credit load.

    "},{"location":"resources/","title":"Resources","text":""},{"location":"setup/","title":"5. Setup","text":""},{"location":"setup/#create-openai-api-token","title":"Create OpenAI API Token","text":"

    Go to https://platform.openai.com/api-keys and create a new API Key.

    "},{"location":"setup/#format-dynatrace-url","title":"Format Dynatrace URL","text":"

    Make a note of your Dynatrace URL, it should be in the following format:

    https://ENVIRONMENT-ID.live.dynatrace.com\n

    For example:

    https://abc12345.live.dynatrace.com\n
    "},{"location":"setup/#create-dynatrace-token","title":"Create Dynatrace Token","text":"

    In Dynatrace, press Ctrl + k and search for access tokens. Choose the first option.

    "},{"location":"setup/#dt_api_token","title":"DT_API_TOKEN","text":"

    Create an API token with these permissions:

    This token will be used by the OpenTelemetry collector and k6 to send data to Dynatrace. The setup script which runs automatically when the codespace is created also uses this to configure span attribute capture rules in Dynatrace this means the relevant OpenTelemetry span attributes will automatically be stored.

    "},{"location":"setup/#recap","title":"\ud83d\udd01 Recap","text":"

    You should now have 3 pieces of information:

    When you have these pieces of information, you can proceed.

    "},{"location":"startup/","title":"6. Start The Demo","text":""},{"location":"startup/#time-to-fire-it-up","title":"\ud83c\udd99 Time to Fire it up","text":"

    Choose one of the following options to start the codespace:

    "},{"location":"startup/#launch-standard-version","title":"Launch Standard Version","text":""},{"location":"startup/#launch-rag-version","title":"Launch RAG Version","text":"

    Leave the top section blank and provide your values in the Recommended secrets form.

    After the codespaces has started (in a new browser tab), the post creation script should begin. This will install everything and will take a few moments.

    When the script has completed, a success message will briefly be displayed (it is so quick you'll probably miss it) and an empty terminal window will be shown.

    TODO: Needs new image

    You may now proceed.

    "},{"location":"use-demo/","title":"7. Use the Demo","text":""},{"location":"use-demo/#accessing-and-using-demo","title":"Accessing and Using Demo","text":"

    In the codespace, switch to the Ports tab. Right click port 30100 and choose Open in Browser

    A new browser tab will open and you should see the demo.

    "},{"location":"use-demo/#using-llm-based-destination-search","title":"Using LLM-based Destination Search","text":"

    Type the name of a destination (eg. Vienna) into the search bar and click the Advise button.

    "},{"location":"use-demo/#what-happens-next","title":"What Happens Next?","text":"

    If you search for Vienna again, this time, the re1sult will be served from the cache - saving you the roundtrip (time and $) to OpenAI / ChatGPT.

    "},{"location":"use-demo/#customer-feedback","title":"Customer Feedback","text":"

    Click the \ud83d\udc4d and / or \ud83d\udc4e buttons to indicate your satisfaction level of the result.

    Clicking these icons will log a message. This log line is then retrieved and processed using DQL in the \"User Sentiment Analysis\" section of the dashboard.

    "},{"location":"visualise-dt/","title":"Visualising Data in Dynatrace","text":""},{"location":"visualise-dt/#uploading-the-dashboards","title":"Uploading the Dashboards","text":"

    This demo comes with several prebuilt dashboards. Do the following in Dynatrace.

    Repeat this process for all the dashboards inside dynatrace/dashboards/*

    "},{"location":"visualise-dt/#distributed-traces","title":"Distributed Traces","text":"

    The application emits distributed traces which can be viewed in Dynatrace:

    Remember that only the very first requests for a given destination will go out to OpenAI. So expect many many more cached traces than \"live\" traces.

    "},{"location":"visualise-dt/#trace-with-openai","title":"Trace with OpenAI","text":"

    A \"full\" call to OpenAI looks like this. Notice the long call halfway through the trace to openai.chat. These traces take much longer (3 seconds vs. 500ms).

    "},{"location":"visualise-dt/#trace-to-weaviate-cache","title":"Trace to Weaviate Cache","text":"

    A call which instead only hits the on-cluster Weaviate cache looks like this.

    Notice that it is much quicker.

    The response TTL (max time that a cached prompt is considered \"fresh\") is checked and if the response is \"still fresh\" (ie. TTL < stale time) the cached value is returned.

    Notice the cached prompt is 123s. The max age (TTL is (by default) 60 minutes. Therefore the prompt is not outdated and thus returned to the user as valid.

    "},{"location":"visualise-dt/#demo-complete","title":"\ud83c\udf89 Demo Complete","text":"

    The demo is now complete. Continue to cleanup your environment.

    "},{"location":"whats-next/","title":"What's Next?","text":"

    Head to the resources page to see other LLM Observability content.

    "},{"location":"snippets/disclaimer/","title":"Disclaimer","text":"

    Support Policy

    This is a demo project created by the Developer Relations team at Dynatrace, showcasing integrations with open source technologies.

    Support is provided via GitHub issues only. The materials provided in this repository are offered \"as-is\" without any warranties, express or implied. Use them at your own risk.

    "},{"location":"snippets/preview-functionality/","title":"Preview functionality","text":"

    Preview Functionality

    This demo uses functionality that is still in Preview.

    After creating a Dynatrace environment, please sign up for this preview functionality.

    "},{"location":"snippets/view-code/","title":"View code","text":"

    View the Code

    The code for this repository is hosted on GitHub. Click the \"View Code on GitHub\" link above.

    "}]} \ No newline at end of file diff --git a/setup/index.html b/setup/index.html index f94b034..6a07a3f 100755 --- a/setup/index.html +++ b/setup/index.html @@ -431,15 +431,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -643,15 +634,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -706,7 +688,11 @@

    🔁 Recap>> Click here to continue with the exercise#

    +
    + +
    diff --git a/startup/index.html b/startup/index.html index 2501342..14143d7 100755 --- a/startup/index.html +++ b/startup/index.html @@ -433,15 +433,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -607,15 +598,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -650,7 +632,11 @@

    Launch RAG Version

    blank terminal window TODO: Needs new image

    You may now proceed.

    -

    >> Click here to continue with the exercise#

    +
    + +
    diff --git a/use-demo/index.html b/use-demo/index.html index 6df7f44..19c23b5 100755 --- a/use-demo/index.html +++ b/use-demo/index.html @@ -462,15 +462,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -625,15 +616,6 @@ - - -
  • - - - >> Click here to continue with the exercise - - -
    • @@ -673,7 +655,11 @@

    What Happens Next?Customer Feedback#

    Click the 👍 and / or 👎 buttons to indicate your satisfaction level of the result.

    Clicking these icons will log a message. This log line is then retrieved and processed using DQL in the "User Sentiment Analysis" section of the dashboard.

    -

    >> Click here to continue with the exercise#

    +
    + +
    diff --git a/visualise-dt/index.html b/visualise-dt/index.html index 3a134da..f469eb0 100755 --- a/visualise-dt/index.html +++ b/visualise-dt/index.html @@ -495,15 +495,6 @@ - - -
  • - - - >> Cleanup resources to avoid GitHub charges - - -
    • @@ -649,15 +640,6 @@ - - -
  • - - - >> Cleanup resources to avoid GitHub charges - - -
    • @@ -711,7 +693,11 @@

    Trace to Weaviate Cachecached request not stale

    🎉 Demo Complete#

    The demo is now complete. Continue to cleanup your environment.

    -

    >> Cleanup resources to avoid GitHub charges#

    +
    + +
    diff --git a/whats-next/index.html b/whats-next/index.html index cad3c62..4d00e48 100755 --- a/whats-next/index.html +++ b/whats-next/index.html @@ -468,17 +468,6 @@ - - @@ -489,34 +478,6 @@ - - - - @@ -541,23 +502,6 @@ - - - @@ -576,7 +520,11 @@

    What's Next?#

    Head to the resources page to see other LLM Observability content.

    -

    Resources#

    +
    + +