Add comprehensive agentic memory API documentation for OpenSearch 3.3

_ml-commons-plugin/agentic-memory.md

@@ -0,0 +1,209 @@
+---
+layout: default
+title: Agentic memory
+nav_order: 60
+---
+
+# Agentic memory
+**Introduced 3.3**
+{: .label .label-purple }
+
+Agentic memory enables AI agents to learn, remember, and reason over structured information across conversations. Unlike simple conversation memory that only stores message history, agentic memory provides persistent, intelligent storage that helps agents maintain context, learn user preferences, and improve their responses over time.
+
+Using agentic memory, you can build AI agents that can do the following:
+
+- Remember user preferences across multiple conversation sessions
+- Learn from past interactions to provide more personalized responses
+- Maintain conversation context beyond simple message history
+- Store and retrieve factual knowledge extracted from conversations
+- Track agent execution traces for debugging and analysis
+- Organize information across different users, sessions, or agent instances
+
+Currently, agentic memory is designed for integration with external agent frameworks like LangChain and LangGraph. OpenSearch's internal [agents]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/agents/) cannot interact with agentic memory.
+{: .note}
+
+## Memory containers
+
+Agentic memory is organized into _memory containers_ that hold all memory types for a specific use case, such as a chatbot, research assistant, or customer service agent.
+
+Each container can be configured with the following components:
+
+- Text embedding models: For semantic search capabilities.
+- Large language models (LLMs): For inference and knowledge extraction.
+- [Memory processing strategies](#memory-processing-strategies): For defining how memories are processed or extracted.
+- [Namespaces](#namespaces): For partitioning and isolating memories by context, user, agent, or session.
+
+For example, to create a memory container with two strategies, send the following request:
+
+```json
+POST /_plugins/_ml/memory_containers/_create
+{
+  "name": "customer-service-agent",
+  "description": "Memory container for customer service agent",
+  "configuration": {
+    "embedding_model_type": "TEXT_EMBEDDING",
+    "embedding_model_id": "your-embedding-model-id",
+    "llm_id": "your-llm-model-id",
+    "strategies": [
+      {
+        "type": "USER_PREFERENCE",
+        "namespace": ["user_id"]
+      },
+      {
+        "type": "SUMMARY",
+        "namespace": ["user_id", "session_id"]
+      }
+    ]
+  }
+}
+```
+{% include copy-curl.html %}
+
+## Memory types
+
+Each memory container can store four distinct types of memory:
+
+- `sessions` -- Manages conversation sessions and their metadata. Each session represents a distinct interaction context between users and agents, containing session-specific information such as start time, participants, and session state.
+
+- `working` -- Stores active conversation data and structured information that agents use during ongoing interactions. This includes recent messages, current context, agent state, execution traces, and temporary data needed for immediate processing.
+
+- `long-term` -- Contains processed knowledge and facts extracted from conversations over time. When inference is enabled, LLMs extract key insights, user preferences, and important information from working memory and store them as persistent knowledge.
+
+- `history` -- Maintains an audit trail of all memory operations (add, update, delete) across the memory container. This provides a comprehensive log of how memories have evolved and changed over time.
+
+## Payload types
+
+When adding memories, you can specify different payload types:
+
+- `conversational` -- Stores conversational messages between users and assistants.
+- `data` -- Stores structured, non-conversational data such as agent state, checkpoints, or reference information.
+
+To add a conversation memory with inference, send the following request:
+
+```json
+POST /_plugins/_ml/memory_containers/<container_id>/memories
+{
+  "messages": [
+    {
+      "role": "user",
+      "content": "I prefer email notifications over SMS"
+    },
+    {
+      "role": "assistant",
+      "content": "I've noted your preference for email notifications"
+    }
+  ],
+  "namespace": {
+    "user_id": "user123",
+    "session_id": "session456"
+  },
+  "payload_type": "conversational",
+  "infer": true
+}
+```
+{% include copy-curl.html %}
+
+To add agent state data, send the following request:
+
+```json
+POST /_plugins/_ml/memory_containers/<container_id>/memories
+{
+  "structured_data": {
+    "agent_state": "researching",
+    "current_task": "analyze customer feedback",
+    "progress": 0.75
+  },
+  "namespace": {
+    "agent_id": "research-agent-1",
+    "session_id": "session456"
+  },
+  "payload_type": "data",
+  "infer": false
+}
+```
+{% include copy-curl.html %}
+
+## Inference mode
+
+You can control how OpenSearch processes memories using the `infer` parameter:
+
+- `false` (default) -- Stores raw messages and data in `working` memory without LLM processing.
+- `true` -- Uses the configured LLM to extract key information and knowledge from the content.
+
+## Memory processing strategies
+
+Memory containers can use the following _strategies_ to automatically process and organize memories:
+
+- `SEMANTIC` -- Groups related memories by meaning and content similarity.
+- `USER_PREFERENCE` -- Extracts and stores user preferences from conversations.
+- `SUMMARY` -- Creates condensed summaries of conversation content.
+
+Strategies are optional; you can create containers without them for simple storage needs.
+
+## Namespaces
+
+_Namespaces_ organize memories within containers by grouping them with identifiers like `user_id`, `session_id`, or `agent_id`. This allows you to separate memories for different users or conversation sessions.
+
+To search memories by namespace, send the following request:
+
+```json
+GET /_plugins/_ml/memory_containers/<container_id>/memories/long-term/_search
+{
+  "query": {
+    "bool": {
+      "must": [
+        {
+          "term": {
+            "namespace.user_id": "user123"
+          }
+        }
+      ]
+    }
+  }
+}
+```
+{% include copy-curl.html %}
+
+## Example use cases
+
+The following examples demonstrate how you can use agentic memory.
+
+### Personalized chatbot
+
+Create a memory container that learns user preferences over time:
+
+- Store conversations in `working` memory with inference enabled.
+- Extract user preferences into `long-term` memory using the `USER_PREFERENCE` strategy.
+- Use namespaces to separate different users' memories.
+
+### Research assistant agent
+
+Build an agent that accumulates knowledge from research sessions:
+
+- Store research queries and results in `working` memory.
+- Use the `SEMANTIC` strategy to group related research topics.
+- Maintain `history` to track how knowledge evolved over time.
+
+### Customer service agent
+
+Develop an agent that remembers customer interactions:
+
+- Store customer conversations with inference to extract key issues.
+- Use `SUMMARY` strategy to create concise interaction summaries.
+- Organize by customer ID using namespaces.
+
+## Getting started
+
+To implement agentic memory in your agents:
+
+1. **[Create a memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container/)** with appropriate models and strategies.
+2. **[Add memories]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/add-memory/)** during agent interactions.
+3. **[Search and retrieve]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/search-memory/)** relevant memories to inform agent responses.
+4. **[Update memories]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/update-memory/)** as conversations evolve.
+
+For detailed API documentation, see [Agentic Memory APIs]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/).
+
+## Next steps
+
+- Explore [memory container configuration]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container/) options.
+- Review the complete [Agentic Memory API reference]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/).

_ml-commons-plugin/agents-tools/index.md

@@ -12,11 +12,10 @@ redirect_from:
 **Introduced 2.13**
 {: .label .label-purple }
 
-You can automate machine learning (ML) tasks using agents and tools. 
+You can automate machine learning (ML) tasks using agents and tools.
 
 An _agent_ orchestrates and runs ML models and tools. For a list of supported agents, see [Agents]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/agents/).
 
 A _tool_ performs a set of specific tasks. Some examples of tools are the [`VectorDBTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/vector-db-tool/), which supports vector search, and the [`ListIndexTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/list-index-tool/), which executes the List Indices API. For a list of supported tools, see [Tools]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/index/).
 
-You can modify and transform tool outputs using a [processor chain]({{site.url}}{{site.baseurl}}/ml-commons-plugin/processor-chain/).
-
+You can modify and transform tool outputs using a [processor chain]({{site.url}}{{site.baseurl}}/ml-commons-plugin/processor-chain/).

_ml-commons-plugin/agents-tools/tools/agent-tool.md

@@ -104,4 +104,8 @@ The following table lists all tool parameters that are available when running th
 
 Parameter	| Type | Required/Optional | Description	
 :--- | :--- | :--- | :---
-`question` | String | Required | The natural language question to send to the LLM. 
+`question` | String | Required | The natural language question to send to the LLM. 
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/connector-tool.md

@@ -165,4 +165,8 @@ When running the agent, you can define any parameter needed for the API call in
       "request_body": "{ \"number1\":\"${parameters.number1}\", \"number2\":\"${parameters.number2}\" }"
     }
   ]
-```
+```
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md

@@ -167,3 +167,7 @@ The following table lists the available tool parameters for running the agent.
 Parameter	| Type | Required/Optional | Description	
 :--- | :--- | :--- | :---
 `index` | String | Required | The index name. Supports wildcards (for example, `weblogs-*`). If wildcards are used, then the tool fetches mappings from the first resolved index and sends them to the LLM. 
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/data-distribution-tool.md

@@ -195,6 +195,10 @@ The following table lists the available tool parameters for running the agent.
 | `dsl` | String | Optional | A complete raw DSL query as a JSON string. If provided, takes precedence over the `filter` parameter. |
 | `ppl` | String | Optional | A complete PPL statement without time information. Used when `queryType` is `ppl`. |
 
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.
+
 ## Limitations
 
 The Data Distribution tool has the following limitations:

_ml-commons-plugin/agents-tools/tools/index-mapping-tool.md

@@ -117,4 +117,8 @@ The following table lists all tool parameters that are available when running th
 Parameter	| Type | Required/Optional | Description	
 :--- | :--- | :--- | :---
 `question` | String | Required | The natural language question to send to the LLM. 
-`index` | Array | Optional | A comma-delimited list of one or more indexes for which to obtain mapping and setting information. Default is an empty list, which means all indexes.
+`index` | Array | Optional | A comma-delimited list of one or more indexes for which to obtain mapping and setting information. Default is an empty list, which means all indexes.
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/index.md

@@ -31,6 +31,8 @@ Specify a tool by providing its `type`, `parameters`, and, optionally, a `descri
 
 Each tool takes a list of parameters specific to that tool. In the preceding example, the `AgentTool` takes an `agent_id` of the agent it will run. For a list of parameters, see each tool's documentation.
 
+You can also run tools directly without creating an agent using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/), which is useful for testing individual tools or performing standalone operations.
+
 |Tool	| Description	|
 |:---	|:---	|
 |[`AgentTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/agent-tool/)	|Runs any agent. |

_ml-commons-plugin/agents-tools/tools/list-index-tool.md

@@ -130,4 +130,8 @@ The following table lists all tool parameters that are available when running th
 
 Parameter	| Type | Required/Optional | Description	
 :--- | :--- | :--- | :---
-`question` | String | Required | The natural language question to send to the LLM. 
+`question` | String | Required | The natural language question to send to the LLM. 
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/log-pattern-analysis-tool.md

@@ -14,7 +14,7 @@ grand_parent: Agents and tools
 {: .label .label-purple }
 <!-- vale on -->
 
-The `LogPatternAnalysisTool` performs an advanced log analysis by detecting anomalous log patterns and sequences through comparative analysis between baseline and selection time ranges. It supports the following analysis modes: 
+The `LogPatternAnalysisTool` performs an advanced log analysis by detecting anomalous log patterns and sequences through comparative analysis between baseline and selection time ranges. It supports the following analysis modes:
 
 - Log sequence analysis (with trace correlation)
 - Log pattern difference analysis
@@ -192,6 +192,10 @@ The following table lists the available tool parameters for running the agent.
 | `selectionTimeRangeStart` | String | Required | The start time for the analysis target period, in UTC date string format (for example, `2025-06-24 07:50:26`). |
 | `selectionTimeRangeEnd` | String | Required | The end time for the analysis target period, in UTC date string format (for example, `2025-06-24 07:55:56`). |
 
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.
+
 ## Limitations
 
 The Log Pattern Analysis tool has the following limitations:

_ml-commons-plugin/agents-tools/tools/log-pattern-tool.md

@@ -117,4 +117,8 @@ Parameter	| Type | Required/Optional      | Description
 :--- | :--- |:-----------------------| :---
 | `index`   | String | Required for DSL queries | The index to search for pattern analysis. |
 | `input`   | String | Required for DSL queries | A DSL query JSON as a string. If both `input` and `ppl` are provided, `input` (DSL) takes precedence. |
-| `ppl`     | String | Required for PPL queries | A PPL query string. Ignored if `input` is also provided. |
+| `ppl`     | String | Required for PPL queries | A PPL query string. Ignored if `input` is also provided. |
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/ml-model-tool.md

@@ -14,7 +14,7 @@ plugins.ml_commons.rag_pipeline_feature_enabled: true
 {: .label .label-purple }
 <!-- vale on -->
 
-The `MLModelTool` runs a machine learning (ML) model and returns inference results. 
+The `MLModelTool` runs a machine learning (ML) model and returns inference results.
 
 ## Step 1: Create a connector for a model
 
@@ -164,4 +164,8 @@ The following table lists all tool parameters that are available when running th
 
 Parameter	| Type | Required/Optional | Description	
 :--- | :--- | :--- | :---
-`question` | String | Required | The natural language question to send to the LLM. 
+`question` | String | Required | The natural language question to send to the LLM. 
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md

@@ -221,3 +221,8 @@ The following table lists all tool parameters that are available when running th
 Parameter	| Type | Required/Optional | Description	
 :--- | :--- | :--- | :---
 `question` | String | Required | The natural language question to send to the LLM. 
+
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.

_ml-commons-plugin/agents-tools/tools/ppl-tool.md

@@ -12,7 +12,7 @@ grand_parent: Agents and tools
 **Introduced 2.13**
 {: .label .label-purple }
 
-The `PPLTool` translates natural language into a PPL query. The tool provides an `execute` flag to specify whether to run the query. If you set the flag to `true`, the `PPLTool` runs the query and returns the query and the results. 
+The `PPLTool` translates natural language into a PPL query. The tool provides an `execute` flag to specify whether to run the query. If you set the flag to `true`, the `PPLTool` runs the query and returns the query and the results.
 
 ## Prerequisite
 
@@ -201,4 +201,8 @@ Parameter	| Type | Required/Optional | Description
 :--- | :--- | :--- | :---
 `index` | String | Required | The index on which to run the PPL query.
 `question` | String | Required | The natural language question to send to the LLM. 
-`verbose` | Boolean | Optional | Whether to provide verbose output. Default is `false`.
+`verbose` | Boolean | Optional | Whether to provide verbose output. Default is `false`.
+
+## Testing the tool
+
+You can run this tool either as part of an agent workflow or independently using the [Execute Tool API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/execute-tool/). The Execute Tool API is useful for testing individual tools or performing standalone operations.