3.0 // Ruby on Rails Mountable Engine

[emorissettegregoire]

Welcome to Stealth 3.0

The time has come to level up Stealth…

Stealth has served Mav and it's community very well over the last 5 years. However, with the evolution of Mav, the company needs features like native voice streaming, Slack integration, and eventing functionality. Stealth 1.X and 2.X has become unwieldy to maintain and shape to our needs.

Additionally, with the technological advancements in LLM technology, we’re leaving a lot on the table by not utilizing LLMs (and relying solely on NLP) within the Stealth pipeline.

Let’s change that and give back to the community what we believe our version of Conversational AI looks like in the world of Ruby, Rails, and LLMs.

Proposed 3.X

I think there are 3 new core principles that will drive Stealth 3.0. Let's dive in...

1. Stealth as a Ruby on Rails Engine

Instead of having Stealth remain a standalone Ruby rack app, let's transition it to be an Ruby on Rails engine (Mountable Plugin). 

This would allow us to achieve a couple of interesting things:

• The ability to “enable” conversational AI on top of any Ruby on Rails app is fantastic! Especially in this new LLM world where the Rails app can be the vector store. It feels much lighter weight (even more so with the new DSL) and easier to get up and started.

• Via our route mount, e.g., mount Stealth::Engine => '/stealth', we can automatically attach a built-in developer web chat for building your bot. No longer are the days where you need to set up a messaging integration and ngrok just to say “hello world”. Note: This may or may not make V1.

• We’ll be able to leverage the parent Rails app as a vector store for RAG implementation and/or a function caller for existing business logic.

💡 Note: We’d create some sort of generator to create the DSL folder structure in the parent app. Devise is actually a great example of an engine that we can model a lot off of.

2. Redesigned DSL

In an effort to make creating a conversational AI bot easier, Mauricio is created a sample DSL framework (codename B2). It includes new syntax that is more intuitive and a brand new eventing system that will allow developers to create omni-channel (SMS, Voice, Slack, etc) experiences much easier.

The highlights of the new DSL (B2) are:

1. Flow declaration is combined with controllers to simplify the code and make adding new flows easier. This eliminates controllers and flow_state.rb.
2. With flow declarations and controllers combined, flow "decoration" appears right inline with each state definition. This makes it easier to understand the flow between states.
3. Weird pluralization of flow names is eliminated.
4. BotController#route is replaced by a new eventing system that is capable of handling many types of events -- including events from voice and Slack.
5. Replies are no longer YAML-based! Instead, replies are pure Ruby and can more easily be specified inline within the state.

Flows, states, helpers, replies, and events are main concepts in this new world. Click into each section to read a bit more on the example implementation.

The updated flows replace controllers and flow_state.rb. They are defined in a new flows top-level directory within the bot. The state-machine nature of Stealth is preserved, but flows now require a lot less boilerplate since they have been combined.

The updated states are defined directly flow file.

The updated helpers are automatically available for all flows.

The updated replies in our new DSL are completely overhauled and are essentially just ruby files.

Lastly, events are a new concept that replaces BotController#route and are capable of handling many types of events -- including events from voice and Slack. This eventing system is better suited to handling things like reactions, image uploads, and other non-text events.

3. LLM-First Architecture

Intent classification, entity recognition and sentiment analysis are core aspects to what makes a great conversational AI experience. Since it's inception Stealth has placed a heavy emphasis on on NLP technology. It's proven hard to train initially and very fickle to scale. AKA: you retrain one intent and it effects another.

Leveraging a LLM to take over these will allow developers to quickly the LLM scale and developer smarter more capable bots quicker.

The foundation for this will be Spectre our open source LLM prompting template library (with RAG support) that will allow developers to use, edit and create their own prompts to send to the LLM to return back a useable system or conversational response.

For Stealth, we'll create out of the box examples that will be included via our generator files, but I'd expect developers to quickly adapt and change their prompts.

These prompts will vary in structure. For example, for intent classification and entity recognition, we'll use a "few shot learning" prompt and you'll be able to access these in the flow files. For natural language generation and rephrasing we'll use a prompt that has access to chat history and intents you'll be able to access these in the flow and reply files.

Our initial focus will be:

- LLM Intent Classification - This can be used for both in scope responses (handle_response) to handle question responses and out of scope responses (I'm driving)to handle routing.
- LLM Entity Recognition - Let's try to handle dates and times out of the box. However, allowing the templates flexibility to create something specific like "insurance_providers"
- LLM Natural Language Generation (and rephrasing) - My gut tells me we'll create a couple of helpers to expand off say. For example, say_nlg and say_nlg_rephrased("I'm sorry, I didn't understand).

With much ✌️ and 🖤,
Mav Automation Ventures.

[matthewblack]

Some light comments / thoughts.

Also, let's make a good practice to comment thoroughly on our new Stealth work. Nothing is better than seeing well articulated comments in OOS projects.