Add VPA for Mimir ingesters

[Rotfuks]

Motivation

• we don't have a HPA for Mimir ingesters but we still need to adapt resource usage
• we would like to keep at least 3 ingesters per installation
• 3 ingesters seems to fit most installations
• but we currently need to manually tune resources for lots of installations

So we can probably set a VPA to rightsize ingesters.
And keep the "mimir needs to be scaled up" alert for when the VPA is maxed out and it's not yet enough, and we need to add an ingester.

Todo

• Investigate which VPA we could use for mimir
• Setup VPA for mimir

Outcome

• Mimir is at any time automatically right-sized using minimal resources while still being resilient.

[QuantumEnigmaa]

New Mimir release now embeds the VPA for the ingester.