[1/3][Asset Selection] Remove key_substring + support wildcards in key:
#27720
Conversation
salazarm
requested review from
smackesey,
OwenKephart,
schrockn,
clairelin135 and
benpankow
| class KeyWildCardAssetSelection(AssetSelection): | ||
| selected_key_wildcard: str |
There was a problem hiding this comment.
Not really happy with the naming here 🙃
salazarm
changed the title
key:key:
salazarm
changed the title
key:key:
salazarm
changed the title
key:key:
…move-key-substring
|
Similar concerns here about about what we are promising going forward in terms of indexes and searchability. I just want to come into this eyes wide open. cc: @alangenfeld @prha |
| value = self.visit(ctx.value()) | ||
| return AssetSelection.key_substring(value) | ||
| value = self.visit(ctx.keyValue()) | ||
| return KeyWildCardAssetSelection(selected_key_wildcard=value) |
There was a problem hiding this comment.
One question for reviewers, should we attach this to the AssetSelection class (eg. AssetSelection.assetsByWildCardString(value))?
salazarm
requested review from
alangenfeld
and removed request for
smackesey
|
Trying to do it against the db directly has to confront that the keys are stored as a json encoded list of string components, so its a more complex translation than just switching Any search backend that has documents for each asset should easily handle it. The test each thing in memory approach can be done over an iterator thats paginating if we want to avoid having them all in memory at once with I would guess marginal latency trade-off. I don't think we can guarantee its fast in all deployment setups but it should be doable. Nothing meaningfully worse than this initial implementation of regex testing all of them in memory. |
Summary & Motivation
key:_prefix_*_middlefix_*_suffix_How I Tested These Changes
pytest