[BUG] Create detector on datastream

[jhill-cmd]

What is the bug?
Cannot create detector on datastream

[security_analytics_exception] null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>

How can one reproduce the bug?
Create a detector on a datastream (winlogbeat-8.9.1, winlogbeat-8.9.1-*, the error also persist with .ds-winlogbeat-8.9.1-*)

Example:

POST _plugins/_security_analytics/detectors
{
  "type": "detector",
  "detector_type": "windows",
  "name": "TEST001",
  "enabled": true,
  "createdBy": "",
  "schedule": {
    "period": {
      "interval": 10,
      "unit": "MINUTES"
    }
  },
  "inputs": [
    {
      "detector_input": {
        "description": "TEST001",
        "indices": [
          "winlogbeat-8.9.1"
        ],
        "pre_packaged_rules": [
          {
            "id": "56d62ef8-3462-4890-9859-7b41e541f8d5"
          },
          {
            "id": "f88bab7f-b1f4-41bb-bdb1-4b8af35b0470"
          },
          {
            "id": "9c8acf1a-cbf9-4db6-b63c-74baabe03e59"
          }
        ],
        "custom_rules": []
      }
    }
  ],
  "triggers": [
  ]
}

gives

  "error": {
    "root_cause": [
      {
        "type": "security_analytics_exception",
        "reason": "null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>"
      }
    ],
    "type": "security_analytics_exception",
    "reason": "null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>",
    "caused_by": {
      "type": "exception",
      "reason": "org.opensearch.alerting.util.AlertingException: null cannot be cast to non-null type kotlin.collections.MutableMap<kotlin.String, kotlin.Any>"
    }
  },
  "status": 500
}

What is the expected behavior?
Create the detector and notify on security events

What is your host/environment?
the 2.6.0, and 2.11.1 Opensearch cluster runned on Debian GNU/Linux 11

Do you have any additional context?
On index not related to datastream it seems to be created. but the issue is occuring on datastream.

[jhill-cmd]

Noticed this as complement

prd-siem-cluster-node-1    | [2023-12-16T18:20:49,055][INFO ][o.o.a.t.TransportIndexMonitorAction] [prd-siem-cluster-node-1] Creating new monitor: {"monitor":{"type":"monitor","schema_version":1,"name":"TEST-001","monitor_type":"doc_level_monitor","user":{"name":"","backend_roles":[],"roles":[],"custom_attribute_names":[],"user_requested_tenant":null},"enabled":false,"enabled_time":null,"schedule":{"period":{"interval":10,"unit":"MINUTES"}},"inputs":[{"doc_level_input":{"description":"TEST-001","indices":["winlogbeat-8.9.1-2023.12"],"queries":[{"id":"9c8acf1a-cbf9-4db6-b63c-74baabe03e59","name":"9c8acf1a-cbf9-4db6-b63c-74baabe03e59","query":"(winlog.event_id: 8004) AND ((winlog.event_data.Workstation: \"Rdesktop\") OR (winlog.event_data.Workstation: \"Remmina\") OR (winlog.event_data.Workstation: \"Freerdp\") OR (winlog.event_data.Workstation: \"Windows7\") OR (winlog.event_data.Workstation: \"Windows8\") OR (winlog.event_data.Workstation: \"Windows2012\") OR (winlog.event_data.Workstation: \"Windows2016\") OR (winlog.event_data.Workstation: \"Windows2019\"))","tags":["medium","windows","attack.credential_access","attack.t1110"]}]}}],"triggers":[],"last_update_time":1702750849028,"data_sources":{"query_index":".opensearch-sap-windows-detectors-queries","findings_index":".opensearch-sap-windows-findings","findings_index_pattern":"<.opensearch-sap-windows-findings-{now/d}-1>","alerts_index":".opensearch-sap-windows-alerts","alerts_history_index":".opensearch-sap-windows-alerts-history","alerts_history_index_pattern":"<.opensearch-sap-windows-alerts-history-{now/d}-1>","query_index_mappings_by_type":{"text":{"analyzer":"rule_analyzer"}},"findings_enabled":true},"owner":"security_analytics"}}
prd-siem-cluster-node-1    | [2023-12-16T18:20:49,180][ERROR][o.o.a.t.TransportIndexMonitorAction] [prd-siem-cluster-node-1] failed to index doc level queries monitor e9fcc4wBTJLvaV9jCJcf. deleting monitor

[jhill-cmd]

The indices mapping was not consistant