|
| 1 | +--- |
| 2 | +layout: default |
| 3 | +title: Scratchpad tools |
| 4 | +has_children: false |
| 5 | +has_toc: false |
| 6 | +nav_order: 60 |
| 7 | +parent: Tools |
| 8 | +grand_parent: Agents and tools |
| 9 | +--- |
| 10 | + |
| 11 | +<!-- vale off --> |
| 12 | +# Scratchpad tools |
| 13 | +**Introduced 3.3** |
| 14 | +{: .label .label-purple } |
| 15 | +<!-- vale on --> |
| 16 | + |
| 17 | +The scratchpad tools consist of `WriteToScratchPadTool` and `ReadFromScratchPadTool`, which enable agents to store and retrieve intermediate thoughts and results during runtime. These tools serve as temporary memory for a single agent execution session, allowing agents to take notes and store important findings during tool executions. |
| 18 | + |
| 19 | +The scratchpad acts as runtime memory that persists only during a single agent execution. When you call the agent's `_execute` API, a new scratchpad is created for that session. All notes and data, except `persistent_notes`, are cleared when the execution completes, ensuring each execution starts with a fresh scratchpad. |
| 20 | +{: .important} |
| 21 | + |
| 22 | +## Use cases |
| 23 | + |
| 24 | +- **Task decomposition**: Store research plans, intermediate findings, and progress notes during multi-step operations within a single execution. |
| 25 | +- **Temporary state management**: Maintain context and accumulated knowledge during the current agent execution session. |
| 26 | +- **Multi-step workflows**: Save key findings after searches to build comprehensive responses in complex tasks. |
| 27 | +- **Execution planning**: Store and reference step-by-step plans during complex operations. |
| 28 | + |
| 29 | +## Scratchpad lifecycle |
| 30 | + |
| 31 | +The scratchpad follows a simple lifecycle: |
| 32 | + |
| 33 | +1. **Creation**: A new, empty scratchpad is created when an agent execution begins. |
| 34 | +2. **Usage**: During execution, the agent can read from and write to the scratchpad multiple times. |
| 35 | +3. **Cleanup**: The scratchpad is automatically cleared when execution completes. |
| 36 | + |
| 37 | +Each call to the agent's `_execute` API creates a fresh scratchpad, ensuring executions are isolated from each other. |
| 38 | + |
| 39 | +## Best practices |
| 40 | + |
| 41 | +- **Structured notes**: Encourage agents to maintain organized, structured notes in the scratchpad. |
| 42 | +- **Regular updates**: Have agents update the scratchpad after each significant step or finding. |
| 43 | +- **Session awareness**: Remember that scratchpad content is temporary and specific to the current execution. |
| 44 | +- **Efficient usage**: Use the scratchpad for intermediate results that need to be referenced multiple times during execution. |
| 45 | + |
| 46 | +## Example: Building a research agent with scratchpad tools |
| 47 | + |
| 48 | +Use the following steps to build a research agent with scratchpad tools. |
| 49 | + |
| 50 | +### Step 1: Register and deploy a model |
| 51 | + |
| 52 | +Register a conversational model that supports the agent framework. The following example uses Anthropic Claude: |
| 53 | + |
| 54 | +```json |
| 55 | +POST /_plugins/_ml/models/_register?deploy=true |
| 56 | +{ |
| 57 | + "name": "Claude Sonnet for Research Agent", |
| 58 | + "function_name": "remote", |
| 59 | + "description": "Claude model for research agent with scratchpad", |
| 60 | + "connector": { |
| 61 | + "name": "Bedrock Claude Sonnet Connector", |
| 62 | + "description": "Amazon Bedrock connector for Claude Sonnet", |
| 63 | + "version": 1, |
| 64 | + "protocol": "aws_sigv4", |
| 65 | + "parameters": { |
| 66 | + "region": "us-east-1", |
| 67 | + "service_name": "bedrock", |
| 68 | + "model": "anthropic.claude-3-5-sonnet-20241022-v2:0" |
| 69 | + }, |
| 70 | + "credential": { |
| 71 | + "access_key": "${AWS_ACCESS_KEY_ID}", |
| 72 | + "secret_key": "${AWS_SECRET_ACCESS_KEY}", |
| 73 | + "session_token": "${AWS_SESSION_TOKEN}" |
| 74 | + }, |
| 75 | + "actions": [ |
| 76 | + { |
| 77 | + "action_type": "predict", |
| 78 | + "method": "POST", |
| 79 | + "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/${parameters.model}/converse", |
| 80 | + "headers": { |
| 81 | + "content-type": "application/json" |
| 82 | + }, |
| 83 | + "request_body": """{"system": [{"text": "${parameters.system_prompt}"}], "messages": ${parameters.messages}, "inferenceConfig": {"maxTokens": 8000, "temperature": 0}}""" |
| 84 | + } |
| 85 | + ] |
| 86 | + } |
| 87 | +} |
| 88 | +``` |
| 89 | +{% include copy-curl.html %} |
| 90 | + |
| 91 | +### Step 2: Register an agent with scratchpad tools |
| 92 | + |
| 93 | +Register a conversational agent that includes both scratchpad tools and other research tools: |
| 94 | + |
| 95 | +```json |
| 96 | +POST /_plugins/_ml/agents/_register |
| 97 | +{ |
| 98 | + "name": "Research Agent with Scratchpad", |
| 99 | + "type": "conversational", |
| 100 | + "description": "Research assistant with persistent scratchpad memory", |
| 101 | + "app_type": "rag", |
| 102 | + "llm": { |
| 103 | + "model_id": "your-model-id", |
| 104 | + "parameters": { |
| 105 | + "max_iteration": 50, |
| 106 | + "system_prompt": "You are a sophisticated research assistant with access to OpenSearch indices and a persistent scratchpad for note-taking.\n\nYour Research Workflow:\n1. Check Scratchpad: Before starting a new research task, check your scratchpad to see if you have any relevant information already saved\n2. Create Research Plan: Create a structured research plan\n3. Write to Scratchpad: Save the research plan and any important information to your scratchpad\n4. Use Search: Gather information using OpenSearch search queries\n5. Update Scratchpad: After each search, update your scratchpad with new findings\n6. Iterate: Repeat searching and updating until you have comprehensive information\n7. Complete Task: Provide a thorough response based on your accumulated research\n\nRemember: Your scratchpad is temporary memory for this execution session only. Use it to organize your thoughts and findings during this task.", |
| 107 | + "prompt": "${parameters.question}" |
| 108 | + } |
| 109 | + }, |
| 110 | + "memory": { |
| 111 | + "type": "conversation_index" |
| 112 | + }, |
| 113 | + "parameters": { |
| 114 | + "_llm_interface": "bedrock/converse/claude" |
| 115 | + }, |
| 116 | + "tools": [ |
| 117 | + { |
| 118 | + "type": "SearchIndexTool" |
| 119 | + }, |
| 120 | + { |
| 121 | + "type": "ListIndexTool" |
| 122 | + }, |
| 123 | + { |
| 124 | + "type": "IndexMappingTool" |
| 125 | + }, |
| 126 | + { |
| 127 | + "type": "ReadFromScratchPadTool", |
| 128 | + "name": "ReadFromScratchPadTool", |
| 129 | + "parameters": { |
| 130 | + "persistent_notes": "You are a helpful researcher. Before making searches, use the ListIndexTool to discover available indices. Write down important notes after using tools." |
| 131 | + } |
| 132 | + }, |
| 133 | + { |
| 134 | + "type": "WriteToScratchPadTool", |
| 135 | + "name": "WriteToScratchPadTool" |
| 136 | + } |
| 137 | + ] |
| 138 | +} |
| 139 | +``` |
| 140 | +{% include copy-curl.html %} |
| 141 | + |
| 142 | +### Step 3: Execute the agent |
| 143 | + |
| 144 | +Execute the agent with a research question: |
| 145 | + |
| 146 | +```json |
| 147 | +POST /_plugins/_ml/agents/<your-agent-id>/_execute?async=true |
| 148 | +{ |
| 149 | + "parameters": { |
| 150 | + "question": "How many residents are in New York?" |
| 151 | + } |
| 152 | +} |
| 153 | +``` |
| 154 | +{% include copy-curl.html %} |
| 155 | + |
| 156 | + |
| 157 | +The agent will: |
| 158 | +1. Read from its scratchpad to check for existing relevant information (starts empty for new executions). |
| 159 | +2. Create and save a research plan to the scratchpad. |
| 160 | +3. Execute searches and update the scratchpad with findings. |
| 161 | +4. Provide a comprehensive answer based on accumulated research. |
| 162 | + |
| 163 | +When using the `agents/<your-agent-id>/_execute` API, you will get a `parent_interaction_id` and `memory_id` in the response. Note the `parent_interaction_id` for later tracing steps. For more information, see [Viewing scratchpad activity](#viewing-scratchpad-activity). |
| 164 | + |
| 165 | +## Tool parameters |
| 166 | + |
| 167 | +The following are the parameters for the scratchpad tools. |
| 168 | + |
| 169 | +### ReadFromScratchPadTool |
| 170 | + |
| 171 | +The following are the **registration parameters** used when adding to an agent. |
| 172 | + |
| 173 | +Parameter | Type | Required/Optional | Description |
| 174 | +:--- | :--- | :--- | :--- |
| 175 | +`persistent_notes` | String | Optional | Initial notes or instructions to store in the scratchpad when first created. |
| 176 | + |
| 177 | +The following are the **execution parameters** used when calling the tool directly. |
| 178 | + |
| 179 | +Parameter | Type | Required/Optional | Description |
| 180 | +:--- | :--- |:------------------| :--- |
| 181 | +`persistent_notes` | String | Required | Initial notes or instructions to store in the scratchpad. |
| 182 | + |
| 183 | +### WriteToScratchPadTool |
| 184 | + |
| 185 | +The following **registration parameters** are used when adding to an agent. |
| 186 | + |
| 187 | +Parameter | Type | Required/Optional | Description |
| 188 | +:--- | :--- | :--- | :--- |
| 189 | +`return_history` | Boolean | Optional | When set to `true`, returns the full scratchpad content after writing. When `false` or omitted (default), returns the newly added note with confirmation. |
| 190 | + |
| 191 | +The following **execution parameters** are used when calling the tool directly. |
| 192 | + |
| 193 | +Parameter | Type | Required/Optional | Description |
| 194 | +:--- | :--- | :--- | :--- |
| 195 | +`notes` | String | Required | The content to write to the scratchpad. |
| 196 | +`return_history` | Boolean | Optional | When set to `true`, returns the full scratchpad content after writing. When `false` or omitted (default), returns the newly added note with confirmation. |
| 197 | + |
| 198 | + |
| 199 | +## Testing the tools |
| 200 | + |
| 201 | +You can use the Tools API directly to execute both scratchpad tools and test their responses before registering them with your agents. |
| 202 | + |
| 203 | +### Testing the ReadFromScratchPadTool |
| 204 | + |
| 205 | +```json |
| 206 | +POST /_plugins/_ml/tools/_execute/ReadFromScratchPadTool |
| 207 | +{ |
| 208 | + "parameters": { |
| 209 | + "persistent_notes": "You are a helpful researcher to conduct searches in OpenSearch cluster. Before making the search, please remember to use the listIndexTool to figure out what are the available indices first. When using listIndexTool, remember the index name has to be in an array format. Please write down important notes after tool used." |
| 210 | + } |
| 211 | +} |
| 212 | +``` |
| 213 | +{% include copy-curl.html %} |
| 214 | + |
| 215 | +When provided `persistent_notes`, the tool attempts to show the persistent notes in the response: |
| 216 | + |
| 217 | +```json |
| 218 | +{ |
| 219 | + "inference_results": [ |
| 220 | + { |
| 221 | + "output": [ |
| 222 | + { |
| 223 | + "name": "response", |
| 224 | + "result": """Notes from scratchpad: |
| 225 | +- You are a helpful researcher to conduct searches in OpenSearch cluster. Before making the search, please remember to use the listIndexTool to figure out what are the available indices first. When using listIndexTool, remember the index name has to be in an array format. Please write down important notes after tool used.""" |
| 226 | + } |
| 227 | + ] |
| 228 | + } |
| 229 | + ] |
| 230 | +} |
| 231 | +``` |
| 232 | + |
| 233 | +You can also test with an empty `persistent_notes` field: |
| 234 | + |
| 235 | +```json |
| 236 | +POST /_plugins/_ml/tools/_execute/ReadFromScratchPadTool |
| 237 | +{ |
| 238 | + "parameters": { |
| 239 | + "persistent_notes": "" |
| 240 | + } |
| 241 | +} |
| 242 | +``` |
| 243 | +{% include copy-curl.html %} |
| 244 | + |
| 245 | +The response indicates that the scratchpad is empty: |
| 246 | + |
| 247 | +```json |
| 248 | +{ |
| 249 | + "inference_results": [ |
| 250 | + { |
| 251 | + "output": [ |
| 252 | + { |
| 253 | + "name": "response", |
| 254 | + "result": "Scratchpad is empty." |
| 255 | + } |
| 256 | + ] |
| 257 | + } |
| 258 | + ] |
| 259 | +} |
| 260 | +``` |
| 261 | + |
| 262 | +### Testing the WriteToScratchPadTool |
| 263 | + |
| 264 | +You can use the Tools API directly to execute the `WriteToScratchPadTool` and test the tool response before registering it with your agents. |
| 265 | + |
| 266 | +```json |
| 267 | +POST /_plugins/_ml/tools/_execute/WriteToScratchPadTool |
| 268 | +{ |
| 269 | + "parameters": { |
| 270 | + "notes": "Research Plan for OpenSearch History and ML Evolution:\\n\\n1. OpenSearch version history, major releases after v2.0\\n2. For each major release:\\n a. Key architectural upgrades\\n b. New machine learning capabilities, especially ML Commons Agent framework \\n c. Descriptions of major Agent tools added\\n d. GitHub issue IDs tied to Agent framework features\\n3. Look for official OpenSearch documentation, release notes, blogs\\n4. Search code repositories for more technical details on ML changes\\n" |
| 271 | + } |
| 272 | +} |
| 273 | +``` |
| 274 | +{% include copy-curl.html %} |
| 275 | + |
| 276 | +The following is the example response from the tool output: |
| 277 | + |
| 278 | +```json |
| 279 | +{ |
| 280 | + "inference_results": [ |
| 281 | + { |
| 282 | + "output": [ |
| 283 | + { |
| 284 | + "name": "response", |
| 285 | + "result": "Wrote to scratchpad: Research Plan for OpenSearch History and ML Evolution:\\n\\n1. OpenSearch version history, major releases after v2.0\\n2. For each major release:\\n a. Key architectural upgrades\\n b. New machine learning capabilities, especially ML Commons Agent framework \\n c. Descriptions of major Agent tools added\\n d. GitHub issue IDs tied to Agent framework features\\n3. Look for official OpenSearch documentation, release notes, blogs\\n4. Search code repositories for more technical details on ML changes\\n" |
| 286 | + } |
| 287 | + ] |
| 288 | + } |
| 289 | + ] |
| 290 | +} |
| 291 | +``` |
| 292 | + |
| 293 | +You can set the `return_history` parameter to `true` to get the full scratchpad content after writing: |
| 294 | + |
| 295 | +```json |
| 296 | +POST /_plugins/_ml/tools/_execute/WriteToScratchPadTool |
| 297 | +{ |
| 298 | + "parameters": { |
| 299 | + "notes": "Research Plan for OpenSearch History and ML Evolution:\\n\\n1. OpenSearch version history, major releases after v2.0\\n2. For each major release:\\n a. Key architectural upgrades\\n b. New machine learning capabilities, especially ML Commons Agent framework \\n c. Descriptions of major Agent tools added\\n d. GitHub issue IDs tied to Agent framework features\\n3. Look for official OpenSearch documentation, release notes, blogs\\n4. Search code repositories for more technical details on ML changes\\n", |
| 300 | + "return_history": true |
| 301 | + } |
| 302 | +} |
| 303 | +``` |
| 304 | +{% include copy-curl.html %} |
| 305 | + |
| 306 | +The response contains the full scratchpad content: |
| 307 | + |
| 308 | +```json |
| 309 | +{ |
| 310 | + "inference_results": [ |
| 311 | + { |
| 312 | + "output": [ |
| 313 | + { |
| 314 | + "name": "response", |
| 315 | + "result": """Scratchpad updated. Full content: |
| 316 | +- Research Plan for OpenSearch History and ML Evolution:\n\n1. OpenSearch version history, major releases after v2.0\n2. For each major release:\n a. Key architectural upgrades\n b. New machine learning capabilities, especially ML Commons Agent framework \n c. Descriptions of major Agent tools added\n d. GitHub issue IDs tied to Agent framework features\n3. Look for official OpenSearch documentation, release notes, blogs\n4. Search code repositories for more technical details on ML changes\n""" |
| 317 | + } |
| 318 | + ] |
| 319 | + } |
| 320 | + ] |
| 321 | +} |
| 322 | +``` |
| 323 | + |
| 324 | +## Viewing scratchpad activity |
| 325 | + |
| 326 | +You can monitor how the agent uses the scratchpad by examining the execution traces: |
| 327 | + |
| 328 | +```json |
| 329 | +GET /_plugins/_ml/memory/message/<parent_interaction_id>/traces?next_token=0 |
| 330 | +``` |
| 331 | +{% include copy-curl.html %} |
| 332 | + |
| 333 | +The traces show the sequence of scratchpad reads and writes, demonstrating how the agent accumulates knowledge during the execution session. |
| 334 | + |
| 335 | +## Related documentation |
| 336 | + |
| 337 | +- [Agents and tools]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/) |
| 338 | +- [Conversational agents]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/agents/) |
0 commit comments