Skip to content

Commit

Permalink
[Security Solution][Detection Engine] fixes IM rule failure when froz…
Browse files Browse the repository at this point in the history
…en tier node is not available (elastic#200621)

## Summary

- addresses elastic/security-team#11117

### How to test


1. Create a deployment with cold and frozen data tiers and use following
commands to create index and ILM

<details>
<summary>Data tiers commands</summary>

```JSON

PUT /_cluster/settings
{
  "persistent": {
    "indices.lifecycle.poll_interval": "10s"
  }
}


PUT /_ilm/policy/filtering_data_tiers
{
  "policy": {
    "phases": {
        "frozen": {
          "min_age": "10s",
          "actions": {
            "searchable_snapshot": {
              "snapshot_repository": "found-snapshots",
              "force_merge_index": true
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "set_priority": {
              "priority": 100
            }
          }
        }
    }
  }
}


PUT /_index_template/filtering_data_tiers_template
{
  "index_patterns": [
    "filtering_data_tiers*"
  ],
  "template": {
    "settings": {
      "index.lifecycle.name": "filtering_data_tiers",
      "index.lifecycle.rollover_alias": "test-filtering_data_tiers"
    },
    "mappings": {
      "_meta": {
        "version": "1.6.0"
      },
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "host": {
          "properties": {
            "name": {
              "type": "keyword",
              "ignore_above": 1024
            }
          }
        }
      }
    }
  }
}

PUT /filtering_data_tiers-000001
{
  "aliases": {
    "filtering_data_tiers": {
      "is_write_index": true
    }
  }
}


POST filtering_data_tiers/_doc
{
  "@timestamp": "2024-07-08T17:00:01.000Z",
  "host.name": "test-0"
}


```

</details>

2. Wait until document moves to frozen tier
3. Run another set of commands to persist document in hot tier

<details>
<summary>Data tiers commands</summary>


```JSON

PUT /_ilm/policy/filtering_data_tiers
{
  "policy": {
    "phases": {
        "frozen": {
          "min_age": "100h",
          "actions": {
            "searchable_snapshot": {
              "snapshot_repository": "found-snapshots",
              "force_merge_index": true
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "set_priority": {
              "priority": 100
            }
          }
        }
    }
  }
}


PUT /filtering_data_tiers-000002
{
  "aliases": {
    "filtering_data_tiers": {
      "is_write_index": true
    }
  }
}


POST filtering_data_tiers/_doc
{
  "@timestamp": "2024-11-08T17:00:01.000Z",
  "host.name": "test-1"
}


```
</details>

4. Pause frozen tier node (admin permissions needed for this) or
increase memory of it, forcing node to become unavailable for short
period of time.
5. Run IM rule with [advanced
setting](https://www.elastic.co/guide/en/security/current/advanced-settings.html#exclude-cold-frozen-data-rule-executions)
filtering out frozen data tier
6. Rule should not fail and generate an alert from document in a hot
tier

---------

Co-authored-by: kibanamachine <[email protected]>
  • Loading branch information
vitaliidm and kibanamachine authored Nov 20, 2024
1 parent b320a37 commit ee397d6
Showing 1 changed file with 4 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,10 @@ export const createThreatSignals = async ({
await services.scopedClusterClient.asCurrentUser.openPointInTime({
index: threatIndex,
keep_alive: THREAT_PIT_KEEP_ALIVE,
// @ts-expect-error client support this option, but it is not documented and typed yet, but we need this fix in 8.16.2.
// once support added we should remove this expected type error
// https://github.com/elastic/elasticsearch-specification/issues/3144
allow_partial_search_results: true,
})
).id;
const reassignThreatPitId = (newPitId: OpenPointInTimeResponse['id'] | undefined) => {
Expand Down

0 comments on commit ee397d6

Please sign in to comment.