The general goal of PerfectFit is to Develop an eCoach app that will provide personalised assistance on smoking cessation and increasing physical activity based on personal (sensor) data (read more here).
This document describes the intended architecture for the eCoach that will be used in the Perfect Fit project. See this document for the design about a more generic version of the eCoach.
- No need for scalability. We expect max 500 participants for the research that will be conducted with the system, thus probably only max 50 users simultaneously using the app. Scalability is nice-to-have for generalisation of the system, but not critical.
- Privacy & Security. Sensitive data such as sensor data and conversations will flow through the app. Privacy and security are critical.
- No need for high performance. Unless it results in an awful user experience
- Generalization. We want the virtual coach system as well as individual components to be reusable outside of the PerfectFit project. So individual components should be modular, and Niceday should be swappable with any other chat framework.
See this overview
The user interacts with the NiceDay smartphone app. This app is developed by sense-health which is part of the consortium.
Its main features are:
- Trackers: simple panels that track amongst others: mood, number of cigarettes smoked, step count, thought records.
- Daily planner: overview of planned actions
- Messaging
A separate app for collecting sensor data that is needed to estimate physical capacity.
- The Niceday app itself cannot gather this data.
- At the start of an activity the sensor data collector is triggered to start recording sensor data.
- After an activity or upon Wifi connection the data is pushed to the Sensor Data Database.
- Data is exposed by a REST API
- Will be based on an existing app and developed outside of scope of PerfectFit virtual coach system.
- Backend for the Niceday app.
- Data storage:
- user profile data,
- app-interaction data
- messages
- Handles user authentication
We cannot change much in the NiceDay app, new PerfectFit features will be implemented through the virtual coach that interacts with the user through the messaging function of the NiceDay app. We will make use of Niceday trackers to gather data, and make use of the Niceday daily planner to plan activities.
A service that pulls data from the Sensor Data Database when needed. Details are deliberately left open, because we will futher co-design this with the Sensor Data Collector developers.
- Validates and processes data
- Writes data directly to the database using the ORM
The conversational agent developed in the Rasa framework.
- Given an input message of the user, perform the appropriate action.
- Actions can range from a simple text message response to something more complex, using any combination of:
- using an algorithm (through calling functions from imported libraries, often with data from db or niceday)
- querying the perfectfit database using ORM
- calling the niceday-api for information from niceday-server
Allows for other components to interact with the agent.
Celery app depending on Redis as message broker. It triggers Rasa to perform actions (mostly starting conversations) through the Rasa REST API at certain points in time when specific conditions are met. In future, the scheduler will also run periodic task not related to Rasa (i.e. running an algorithm, or data preprocessing)
A worker that sends back and forth messages from a niceday user between the Niceday server and the rasa agent.
- Depends on the javascript Niceday client goalie-js.
- Listens to incoming messages, upon an incoming message it sends a request to the rasa agent REST API
- It forwards any text response from rasa back to the user through Niceday
Allows control over certain functionalities in the niceday app. It is a node.js REST API that wraps functions that we need from the javascript Niceday client goalie-js.
- Request data from the niceday server
- Read from and manage Niceday (custom) trackers
- Interact with Niceday daily planner
Stores PerfectFit-specific data,
- Stores:
- Data about the user (name, age, stage of patient journey, preferences)
- Sensor data
- Any data that we need to conduct research
- The interface to it will be an SqlAlchemy ORM (so no API abstraction)
Interface for domain experts to control the content of the application (I.e. what the conversational agent responds with). Together with domain experts in the consortium we will decide on how much content/behavior we want to be managed vs to be hard-coded.
Some options:
- A new tool that we develop ourselves, that allows for full control over the behavior and responses of the agent. Somewhat similar to botpress
- Rasa X, a (proprietary) tool for Conversation-Driven Development (CDD), the process of listening to your users and using those insights to improve your AI assistent. This provides (amongst others) a UI for annotation of conversations, thereby steering the behavior of the agent.
- Rasa agent behavior and training data is fully controlled by developers, but some specific content comes from a CMS. The CMS is basically a mapping from a response identifier (i.e. UTTER_WELCOME) to the actual response (i.e. 'Welcome to this app')
A number of algorithm components
- For example: a Sensor Data Processor, that has as input sensor data for an activity, and outputs some useful information about the activity (i.e. was the capacity threshold reached).
- They are standalone Python libraries that are imported and used by Rasa agent, but can also be used outside of this project.
- Should be setup in a modular and functional way. So no connections to the database or other components, the interface should be functions or methods that take as input all the required data used by the algorithm.
Interface to inspect data, monitor the system, and perform actions in the system.
- Design for this will emerge based on the need for certain control/insights while progressing in the project
Interface to schedule periodic tasks by taking advantage of Django admin.
It could be that the algorithsm processing (or preprocessing before sending to algorithms) of sensor or user data will be too compute-heavy. To solve that we can have dedicated workers that process data. The system should then get some more queuing, i.e. the sensor data REST api just writes data to a queue. The workers are subscribed to this queue and process the data, then notify other components or write to the database.
Some data might only be needed for a certain time period, especially sensor data. A specific worker should remove data if it is expired (data should have an expiry-date field in db).
See it also on whimsical
An overview of the functionalities built into PerfectFit is shown here below. These repositories implement the chatbot conversation functionality.
The chatbot uses a set of components that allow the conversation to flow in the intended way, consuming and saving data from different sources (i.e., Niceday server and local DB).
The conversaton steps start with a text message sent by the user. The message is received by the Broker, that sends a request to the Rasa engine.
Rasa produces a text response to the message received, and sends it back to the user as a list of text messages.
A representation of the interactions among the components in the conversation flow is shown here below:
The communication between the user and the PerfectFit chatbot makes use of the goalie-js module for messages exchange with the Niceday Server, in combination with the niceday-broker.
When the niceday-broker package is built and run (see here), it connects to the Niceday Server with a therapist account using the credentials stored in the .env file contained in the virtual-coach-main repository.
Upon a successful connection, the broker subscribes to the incoming messages directed to the therapist account, and forwards them to the Rasa conversational engine.
Rasa responds to the user’s messages as a therapist, following the conversation logic implemented in Rasa_Bot.
When a new message is received, the broker composes a message according to the rasa server interface specifications and sends it to the Rasa server though an HTTP POST request.
The Rasa Bot determines the response to the message sent by the Broker, and sends it back as a list of text messages in the response to the HTTP request.
Upon the response reception, the broker sends each single message to the Niceday server using the sendTextMessage(id, text) function of the Chat object of goalie-js.
In the following diagram, a representation of the messaging sequence is shown.
As an example, in the diagram the execution of two actions and the production of two response messages is shown.
To retrieve data from the Niceday Server, the Niceday API has to be used. The API is used by the Rasa actions, using the NicedayClient Python interface.
User’s data and output of the conversations can be stored to and retrieved from the local DB. The interactions with the DB are performed by the rasa actions, using the virtual-coach-db package.
The different components exchange information one with the other using HTTP requests. The requests specifications are reported here below.
N.B. The URLs reported are assuming that the component is running using the Docker image generated using the docker-compose file.
If the components are running in isolation, or outside the Docker network, the URL has to be changed accordingly. For example, to set the Niceday API endpoints to localhost, it should be changed to http://localhost:8080/.
URL: http://rasa_server:5005/webhooks/rest/webhook
Request type: POST
Body:
{
"sender":"sender_id",
"message":"text_message"
}
Output:
[
{
"recipient_id": "id",
"text": "text_message"
},
{
"recipient_id": "id",
"text": " text_message”
}
]
URL: http://rasa_actions:5005/webhook
URL: http://niceday_api:8080/
Specifications for the API usage can be found in the Swagger UI interface, as shown here.
The Broker cannot be queried directly, but the goalie-js interfaces have to be used.