diff --git a/docs/apis/plugintypes/ai/index.md b/docs/apis/plugintypes/ai/index.md new file mode 100644 index 0000000000..1f1002c147 --- /dev/null +++ b/docs/apis/plugintypes/ai/index.md @@ -0,0 +1,39 @@ +--- +title: AI Plugins +tags: + - AI + - LLM + - Provider + - Placement +--- + +The AI subsystem in the LMS is designed to be extensible, allowing for the integration of external AI services. +This is achieved through the use of AI plugins, which are divided into two types: Providers and Placements. + +### Placements + +The aim of Placements is to provide a consistent UX and UI for users when they use AI backed functionality. + +Placement plugins leverage the functionality of the other components of the AI subsystem. +This means plugin authors can focus on how users interact with the AI functionality, without needing to +implement the AI functionality itself. + +Because Placements are LMS plugins in their own right and are not "other" types of LMS plugins, +it gives great flexibility in how AI functionality is presented to users. + +See the [Placements](/apis/plugintypes/ai/placement.md) documentation for more information +on developing Placement plugins. + +### Providers + +Provider plugins are the interface between the LMS AI subsystem and external AI systems. +Their focus is on converting the data requested by an Action into the format needed by the +external AI services API, and then correctly providing the response back from the AI +in an Action Response object. + +Because of this design the Providers that provide the AI Actions can be swapped out, mix and matched +or upgraded; both without the need to update the Placement code and without the need to change the +way users interact with the functionality. + +See the [Providers](/apis/plugintypes/ai/provider.md) documentation for more information +on developing Provider plugins. diff --git a/docs/apis/plugintypes/ai/placement.md b/docs/apis/plugintypes/ai/placement.md new file mode 100644 index 0000000000..a157bb3841 --- /dev/null +++ b/docs/apis/plugintypes/ai/placement.md @@ -0,0 +1,129 @@ +--- +title: Placements +tags: + - AI + - LLM + - Placement +--- + +The aim of Placements is to provide a consistent UX and UI for users when they use AI backed functionality. + +Placement plugins leverage the functionality of the other components of the AI subsystem. +This means plugin authors can focus on how users interact with the AI functionality, without needing to +implement the AI functionality itself. + +Because Placements are LMS plugins in their own right and are not "other" types of LMS plugins, +it gives great flexibility in how AI functionality is presented to users. + +:::warning The Golden Rule: + +Placements DO NOT know about Providers, and Providers DO NOT know about Placements. +Everything should go via the Manager. + +::: + +Placements are defined as classes in their own namespace according to their plugin name. +The naming convention for Action classes is `aiplacement_`, +for example: `aiplacement_editor`. With corresponding namespaces. + +Each Placement MUST inherit from the `\core_ai\placement` abstract class. +They must also implement the following methods: + +- `get_action_list(): array` This is the list of Actions that are supported by this Placement, for example the `aiplacement_editor` plugin defines this as: + +```php +public function get_action_list(): array { + return [ + \core_ai\aiactions\generate_text::class, + \core_ai\aiactions\generate_image::class, + ]; +} +``` + +## Capabilities and Permissions + +Placements are responsible for determining who and where a Placement (and by extension an Action can be used). +It is not the job of Actions or Providers to determine access. + +## Action Processing + +The following is the basic workflow in order for a placement to have an action processed for a user request: + +- The Placement instantiates a new action object of type they wish to use. +- The action must be instantiated and passing it the required data. Each action will define what configuration it needs. As an example: + +```php +// Prepare the action. +$action = new \core_ai\aiactions\generate_image( + contextid: $contextid, + userid: $USER->id, + prompttext: $prompttext, + quality: $quality, + aspectratio: $aspectratio, + numimages: $numimages, + style: $style, +); +``` + +- The Placement then instantiates the Manager class and calls `process_action()` +- passing in the configured action object: + +```php +// Send the action to the AI manager. +$manager = \core\di::get(\core_ai\manager::class); +$response = $manager->process_action($action); +```` + +- The process_action() method will then return a response object (instance of `responses\response_base`). +- It is up to the Placement to check for success (or not) of the response and pass the result back to the + user or for further processing. + +## Plugin Structure + +Placement plugins reside in the `ai/placement` directory. + +Each Placement is in a separate subdirectory and consists of a number of mandatory files and any other +files the developer is going to use. + +The following is the typical structure of a Placement plugin, using the Editor Placement as an example: + +```bash +. +├── classes +│   ├── external +│   │   ├── generate_image.php +│   │   └── generate_text.php +│   ├── placement.php +│   └── privacy +│   └── provider.php +├── db +│   ├── access.php +│   └── services.php +├── lang +│   └── en +│   └── aiplacement_editor.php +└── version.php + +``` + +## Settings + +Settings for the Placement should be defined in the `settings.php` file. +Each Placement plugin should create a new admin settings page using `core_ai\admin\admin_settingspage_provider` class. + +This is the same as for Provider plugins, for example: + +```php +use core_ai\admin\admin_settingspage_provider; + +if ($hassiteconfig) { + // Placement specific settings heading. + $settings = new admin_settingspage_provider( + 'aiprovider_openai', + new lang_string('pluginname', 'aiprovider_openai'), + 'moodle/site:config', + true, + ); + +... +``` diff --git a/docs/apis/plugintypes/ai/provider.md b/docs/apis/plugintypes/ai/provider.md new file mode 100644 index 0000000000..1af3f249c2 --- /dev/null +++ b/docs/apis/plugintypes/ai/provider.md @@ -0,0 +1,228 @@ +--- +title: Providers +tags: + - AI + - LLM + - Provider +--- +Providers are the interface between the LMS AI subsystem and external AI systems. +Their focus should be on converting the data requested by an Action into the format needed +by the external AI services API, and then correctly providing the response back from the AI +in an Action Response object. + +:::warning The Golden Rule: + +Placements DO NOT know about Providers, and Providers DO NOT know about Placements. +Everything should go via the Manager. + +::: + +Providers are defined as classes in their own namespace according to their plugin name. +The naming convention for Action classes is `aiprovider_`, +for example: `aiprovider_openai`, `aiprovider_azureai`. With corresponding namespaces. + +Each Provider MUST inherit from the `\core_ai\provider` abstract class. +They must also implement the following methods: + +- `get_action_list(): array` This is the list of Actions that are supported by this Provider, for example the `aiprovider_openai` plugin defines this as: + +```php +public function get_action_list(): array { + return [ + \core_ai\aiactions\generate_text::class, + \core_ai\aiactions\generate_image::class, + \core_ai\aiactions\summarise_text::class, + ]; +} +``` + +## Process classes + +For each action supported by the provider, the provider plugin **MUST** implement a `process_` class, +where `` is the name of the action. For example: `process_generate_image`. + +Every process action class **MUST** inherit from the `\core_ai\process_base` abstract class. + +The process action class **MUST** implement a `process()` method. This method is responsible for +converting the data requested by an Action into the format needed by the external AI services API, +and then correctly providing the response back from the AI in an Action Response object. + +The process action classes and process method are expected by the manager to exist and be callable. + +As most provider plugins will support more than one action, it is recommended to create an +`abstract_processor` class that inherits from the `\core_ai\process_base` class and then have each +process action class inherit from this abstract class. + +For example, the `aiprovider_openai` plugin defines an `abstract_processor` class that inherits from +the `\core_ai\process_base` class and then the `process_generate_image`, `process_generate_text` and +`process_summarise_text` classes inherit from this abstract class. + +This can be visualised as follows: + +```mermaid +graph TD; + A[process_base] --> B[abstract_processor] + B[abstract_processor] --> C[process_generate_image] + B --> D[process_generate_text] + B --> E[process_summarise_text] + ``` + +Apart from this, Providers are free to define their own structure. It should be kept in mind that Providers +are designed to be a "thin wrapper" around the external AI systems API. They shouldn't store data, +or have their own UI elements (beyond what is required for configuration). + +## Plugin Structure + +Provider plugins reside in the `ai/provider` directory. + +Each Provider is in a separate subdirectory and consists of a number of mandatory files and any other +files the developer is going to use. + +The following is the typical structure of a Provider plugin, using the OpenAI Provider as an example: + +```bash +. +├── classes +│   ├── abstract_processor.php +│   ├── privacy +│   │   └── provider.php +│   ├── process_generate_image.php +│   ├── process_generate_text.php +│   ├── process_summarise_text.php +│   └── provider.php +├── lang +│   └── en +│   └── aiprovider_openai.php +├── settings.php +├── tests +│   ├── fixtures +│   │   ├── image_request_success.json +│   │   ├── test.jpg +│   │   └── text_request_success.json +│   ├── process_generate_image_test.php +│   ├── process_generate_text_test.php +│   ├── process_summarise_text_test.php +│   └── provider_test.php +└── version.php + +``` + +## Settings + +Settings for the Provider should be defined in the `settings.php` file. +Each Provider plugin should create a new admin settings page using `core_ai\admin\admin_settingspage_provider` class. + +For example, the `aiprovider_openai` plugin defines this: + +```php +use core_ai\admin\admin_settingspage_provider; + +if ($hassiteconfig) { + // Provider specific settings heading. + $settings = new admin_settingspage_provider( + 'aiprovider_openai', + new lang_string('pluginname', 'aiprovider_openai'), + 'moodle/site:config', + true, + ); + +... +``` + +## Rate Limiting + +It is recommended that Providers implement rate limiting to prevent abuse of the external AI services. + +To assist with this, the AI subsystem provides a `core_ai\rate_limiter` class that can be used to implement rate limiting. +This class supports both user and system level rate limiting. + +This should be implemented in a `is_request_allowed()` method in the Provider class. For example, from the +`aiprovider_openai` plugin: + +```php +/** + * Check if the request is allowed by the rate limiter. + * + * @param aiactions\base $action The action to check. + * @return array|bool True on success, array of error details on failure. + */ + public function is_request_allowed(aiactions\base $action): array|bool { + $ratelimiter = \core\di::get(rate_limiter::class); + $component = \core\component::get_component_from_classname(get_class($this)); + + // Check the user rate limit. + if ($this->enableuserratelimit) { + if (!$ratelimiter->check_user_rate_limit( + component: $component, + ratelimit: $this->userratelimit, + userid: $action->get_configuration('userid') + )) { + return [ + 'success' => false, + 'errorcode' => 429, + 'errormessage' => 'User rate limit exceeded', + ]; + } + } + + // Check the global rate limit. + if ($this->enableglobalratelimit) { + if (!$ratelimiter->check_global_rate_limit( + component: $component, + ratelimit: $this->globalratelimit + )) { + return [ + 'success' => false, + 'errorcode' => 429, + 'errormessage' => 'Global rate limit exceeded', + ]; + } + } + + return true; + } +``` + +If implementing rate limiting, settings for the rate limits should be provided in the plugin settings. + +For example, the `aiprovider_openai` plugin provides settings for the user and global rate limits: + +```php + // Setting to enable/disable global rate limiting. + $settings->add(new admin_setting_configcheckbox( + 'aiprovider_openai/enableglobalratelimit', + new lang_string('enableglobalratelimit', 'aiprovider_openai'), + new lang_string('enableglobalratelimit_desc', 'aiprovider_openai'), + 0, + )); + + // Setting to set how many requests per hour are allowed for the global rate limit. + // Should only be enabled when global rate limiting is enabled. + $settings->add(new admin_setting_configtext( + 'aiprovider_openai/globalratelimit', + new lang_string('globalratelimit', 'aiprovider_openai'), + new lang_string('globalratelimit_desc', 'aiprovider_openai'), + 100, + PARAM_INT, + )); + $settings->hide_if('aiprovider_openai/globalratelimit', 'aiprovider_openai/enableglobalratelimit', 'eq', 0); + + // Setting to enable/disable user rate limiting. + $settings->add(new admin_setting_configcheckbox( + 'aiprovider_openai/enableuserratelimit', + new lang_string('enableuserratelimit', 'aiprovider_openai'), + new lang_string('enableuserratelimit_desc', 'aiprovider_openai'), + 0, + )); + + // Setting to set how many requests per hour are allowed for the user rate limit. + // Should only be enabled when user rate limiting is enabled. + $settings->add(new admin_setting_configtext( + 'aiprovider_openai/userratelimit', + new lang_string('userratelimit', 'aiprovider_openai'), + new lang_string('userratelimit_desc', 'aiprovider_openai'), + 10, + PARAM_INT, + )); + $settings->hide_if('aiprovider_openai/userratelimit', 'aiprovider_openai/enableuserratelimit', 'eq', 0); +``` diff --git a/docs/apis/subsystems/ai/index.md b/docs/apis/subsystems/ai/index.md new file mode 100644 index 0000000000..2450e02843 --- /dev/null +++ b/docs/apis/subsystems/ai/index.md @@ -0,0 +1,267 @@ +--- +title: AI Subsystem +tags: + - AI + - LLM + - Provider + - Placement +--- + + + +The AI Subsystem in Moodle LMS provides a consistent and user-friendly way for users to interact with AI +in Moodle's user interface, as they do their teaching and learning activities. +As well as providing a straightforward way to integrate with various AI providers on the backend. + +## What is AI in this context? + +When we talk about AI in the context of this subsystem, AI is: anything that's plugged in via a Provider +Plugin and provides one or more Actions. + +Artificial Intelligence (AI), Machine Learning (ML), Large Language Models (LLMs), Generative AI, etc. +are all individual terms with discrete meanings. However, to keep things simple and in the context of +the subsystem everything is just called AI. + +## Design Overview + +The AI subsystem consists of the following (main) components: + +- **Placements** + - Are the UI components and associated workflows that allow users to interact with AI. + - They implement one or more AI actions. + - They provide a consistent experience for users regardless of which AI is providing the action + - They are LMS plugins +- **Actions** + - These are the "things" users can do with AI. + - Examples of an action include: generating text, generating an image, providing a chat interface. + - Actions don't have a UI, they are classes that provide a predictable interface. +- **Providers** + - Providers are the interface between AI systems and the Moodle LMS. + - They implement support for one or more actions. + - Providers should not have an UI, apart from those required for settings, + to configure and manage the provider. For example an Administration UI to allow setting of API + keys; and/or to enable or disable the enabled actions. + - The aim of Providers is to make it "easy" to integrate AI systems with LMS, + without the need to implement a UI to use them. + - They are LMS plugins +- **Subsystem Manager** + - This is implemented at code level and is a core part of the subsystem design. + - It is the "controller" that sits between Placements and Providers. + - It allows Placements to pass user requests to Providers and handles all the calls to + the providers including prioritisation and logging. + - It allows Providers to respond to action requests. + +### Placements + +The aim of Placements is to provide a consistent UX and UI for users when they use AI backed functionality. + +Placement plugins leverage the functionality of the other components of the AI subsystem. +This means plugin authors can focus on how users interact with the AI functionality, without needing to +implement the AI functionality itself. + +Because Placements are LMS plugins in their own right and are not "other" types of LMS plugins, +it gives great flexibility in how AI functionality is presented to users. + +See the [Placements](/apis/plugintypes/ai/placement.md) documentation for more information +on developing Placement plugins. + +### Providers + +Provider plugins are the interface between the LMS AI subsystem and external AI systems. +Their focus is on converting the data requested by an Action into the format needed by the +external AI services API, and then correctly providing the response back from the AI +in an Action Response object. + +Because of this design the Providers that provide the AI Actions can be swapped out, mix and matched +or upgraded; both without the need to update the Placement code and without the need to change the +way users interact with the functionality. + +See the [Providers](/apis/plugintypes/ai/provider.md) documentation for more information +on developing Provider plugins. + +### Subsystem Manager + +The Subsystem Manager is the "controller" that sits between Placements and Providers. + +:::warning The Golden Rule: + +Placements DO NOT know about Providers, and Providers DO NOT know about Placements. +Everything should go via the Manager. + +::: + +The manager class `\core_ai\manager()` is the "controller" for the subsystem. +In general it will be how most processes will interact with the AI subsystem. + +The main method in the class is `process_action(base $action): responses\response_base` +this is the entry point for Action processing. Every Placement that wants to process an AI action +calls this method. +The manager will determine what Providers support this action and then hand off the action object to +a Provider. The Provider **MUST** return an action response object. This method will also store the +result of each action call. + +The manager class also has various utility methods that can be accessed. +Such as getting the list of providers for specific actions, which is used to help render +administration settings. + +### AI User Policy + +Inline with Moodle's AI Principles and as guided by emerging legislation; users must accept an +AI policy before using AI in LMS. As the requirements are different to a site policy +(legislation, seems to indicate it acknowledgement of AI must be made at point of use), +separate policy functionality has been implemented for the subsystem. + +All Placements must implement a check to see if a user has accepted the AI Policy. +Placements must also provide a way for users to accept the policy. +If a user has not previously accepted the AI Policy, the Placement must display the policy to the +user, and the user is not able to use the placement until they accept the policy. +Users only need to accept the policy once. + +To assist Placements with policy display and acceptance the Manager provides the following functionality: + +- The Manager makes the following methods available for Placements to call directly: + - `\core_ai\manger::get_user_policy(int $userid): bool` - + Given a user ID (record id in user table), returns true if the user has accepted the policy, + false otherwise. It handles looking up the status and caching the response. + - `\core_ai\manager::set_user_policy(int $userid, int $contextid): bool` - + Given a user ID and a Context ID (of the context the policy was displayed in) set the policy + acceptance. +- The manager class also makes available webservices to be used for policy setting and getting. + This helps Placements set policy acceptance via ajax as part of a UI workflow: + - `core_ai_get_policy_status` - + Gets the policy status for a user. Calls: `core_ai\external\get_policy_status` + - `core_ai_set_policy_status` - + Sets the policy status for a user. Calls: `core_ai\external\set_policy_status` + +### Actions + +Actions provide a structured way to work with AI. Placements create an Action object when they want +to interact with AI and Providers, and it is an Action that Providers consume. + +Actions are defined as classes in the `\core_ai\aiactions` namespace. +The naming convention for Action classes is `_`, +for example: `generate_image`, `translate_text`. + +Each action **MUST** inherit from the `\core_ai\aiactions\base()` abstract class. +They must also implement two methods: + +- `__construct(...args): void` + - The constructor method is allowed to have a variable signature, so that each action can define its own + configuration requirements. + - The method **MUST** take a `contextid` as one of the variables. + - An Action **MUST** be correctly instantiated before it can be used and passed onto the AI manager. + For example the constructor method for the generate_image Action is: + +```php +public function __construct( + int $contextid, + /** @var int The user id requesting the action. */ + protected int $userid, + /** @var string The prompt text used to generate the image */ + protected string $prompttext, + /** @var string The quality of the generated image */ + protected string $quality, + /** @var string The aspect ratio of the generated image */ + protected string $aspectratio, + /** @var int The number of images to generate */ + protected int $numimages, + /** @var string The visual style of the generated image */ + protected string $style, +) { + parent::__construct($contextid); +} +``` + +- `store(response_base $response): int` + - This method is responsible for storing any action specific data related to the action in the + LMS database. + - Each Action must store its own data can that can be referenced later. + - It takes a matching response class, that contains the result of the action call. + - For example the store() call form the generate_image Action is: + +```php +#[\Override] +public function store(response_base $response): int { + global $DB; + + $responsearr = $response->get_response_data(); + + $record = new \stdClass(); + $record->prompt = $this->prompttext; + $record->numberimages = $this->numimages; + $record->quality = $this->quality; + $record->aspectratio = $this->aspectratio; + $record->style = $this->style; + $record->sourceurl = $responsearr['sourceurl']; // Can be null. + $record->revisedprompt = $responsearr['revisedprompt']; // Can be null. + + return $DB->insert_record($this->get_tablename(), $record); +} +``` + +It is up to the action to define its own database schema and stored data, that is relevant to +what the action does. For example the database table definition for the generate_image Action is: + +```xml + + + + + + + + + + + + + + +
+``` + +The naming convention for Action database tables is `ai_action_`, +for example: `ai_action_generate_image`, `ai_action_translate_text`. + +#### Responses + +When a Provider processes an Action it **MUST** return a response object. +This allows Placements to receive an expected response for any Action call. +Each Action has a matching response class. The provider that processes the Action will instantiate +an instance of this response class and populate it with the data required for this type of response. + +Each Action response MUST inherit from the `\core_ai\aiactions\responses\response_base` abstract +class. They must also implement two methods: + +- `set_response(array $response): void` + - Taking an array of response variables (which must be defined as class variables), + it sets these against class variables so they can be retrieved by the Manager and calling Placement. +- `get_response(): array` + - Returns the set response data. + +For example the `set_response()` for the generate_image Action response is: + +```php + #[\Override] + public function set_response_data(array $response): void { + $this->draftfile = $response['draftfile'] ?? null; + $this->revisedprompt = $response['revisedprompt'] ?? null; + $this->sourceurl = $response['sourceurl'] ?? null; + } +``` + +And the `get_response()` for the generate_image Action response is: + +```php + #[\Override] + public function get_response_data(): array { + return [ + 'draftfile' => $this->draftfile, + 'revisedprompt' => $this->revisedprompt, + 'sourceurl' => $this->sourceurl, + ]; + } +``` + +The naming convention for Action Response classes is `response_`, +for example: `response_generate_image`, `response_translate_text`.