[BUG] script_exception Runtime Error When Previewing Trigger Condition in Per Cluster Metrics Monitor

[getsolaris]

What is the bug?

When creating a Per Cluster Metrics Monitor in the OpenSearch Plugins tab, selecting "Nodes stats" as the request type and clicking on "Preview condition response" after setting the trigger results in a script_exception runtime error.

How can one reproduce the bug?

Steps to reproduce the behavior:

1. Go to OpenSearch Dashboards.
2. Navigate to the "Plugins" tab and click on "Create Monitor."
3. Select "Per Cluster Metrics Monitor" as the monitor type.
4. For the query request type, choose "Nodes stats."
5. Click on "Preview query" to check the query response.
6. Set a trigger and leave the "Trigger condition" as default.
7. Click on "Preview condition response."
8. See the script_exception runtime error.

What is the expected behavior?

The trigger condition preview should return a valid response without throwing a script_exception runtime error.

What is your host/environment?

• OpenSearch Version: OpenSearch 2.15 (OpenSearch_2_15_R20240904-P2)
• �Dashboard Version: v 2.15.0
• OS: Mac OS
• Browser and version: Version 129.0.6668.100 (Official Build) (arm64)

Do you have any screenshots?

• Trigger condition

ctx.results[0].nodes.NODE_ID.jvm.mem.heap_used_percent > 60

• Trigger condition response

{
  "type" : "script_exception",
  "reason" : "runtime error",
  "script_stack" : [
    "ctx.results[0].nodes.NODE_ID.jvm.mem.heap_used_percent > 60",
    "                            ^---- HERE"
  ],
  "script" : "ctx.results[0].nodes.NODE_ID.jvm.mem.heap_used_percent > 60",
  "lang" : "painless",
  "position" : {
    "offset" : 28,
    "start" : 0,
    "end" : 59
  },
  "caused_by" : {
    "type" : "null_pointer_exception",
    "reason" : "Cannot invoke \"Object.getClass()\" because \"callArgs[0]\" is null"
  }
}

Do you have any additional context?

This error only occurs when trying to preview the trigger condition. Query preview works as expected, and the error appears during the preview of the trigger condition response.

[dblock]

[Catch All Triage - 1, 2, 3, 4]

[AWSHurneyt]

Hi @getsolaris,
That error is calling out that the mustache template used for the message content is malformed in some way. The error is calling out that the issue is caused by the NODE_ID portion.

{{ctx.results[0].nodes.NODE_ID.jvm.mem.heap_used_percent}}

The NODE_ID text is just a placeholder in the sample template; you'd want to supply your own node ID, as that's what will be available in the nodes stats response.

You could otherwise consider iterating through the list of nodes in the response. Although it doesn't discuss the nodes/stats API specifically, this blog post discusses iterating through results in cat/indices and cat/shards responses.
https://opensearch.org/blog/cluster-metrics-monitors-blog/#:~:text=are%20%22UNASSIGNED.%22%0A%7B%7B%23ctx.-,results,-.0.shards%7D%7D%0A%2D%20%7B%7Bindex

Here's the related mustache template for reference as well.
https://mustache.github.io/mustache.5.html#:~:text=Shown.-,Non%2DEmpty%20Lists,-If%20the%20person