Migrate from deprecated methods and to new controllers..

[burmanm]

What is missing?

controller-runtime has been updated quite a lot since our last update and has following changes.

• admission.Validator is deprecated and replaced with CustomValidator. We need to move to that.
• ObjectReconciler is created, thus we can reduce the amount of boilerplate from our reconciler
• Verify if some bugged tests can be re-enabled
• "k8s.io/utils/pointer" is deprecated, use k8s.io/utils/ptr
• /metrics can be secured with TLS
• "sigs.k8s.io/controller-runtime/pkg/config/v1alpha1" is deprecated, remove and replace its usage
• We can allow pprof for easier debugging/profiling of cass-operator, check Manager.Options.PprofBindAddress
• Use reconcile.TerminalError(...) instead of just bailing out without error

Why is this needed?

controller-runtime is quite aggressive in removing deprecated methods and as such we need to keep up with the changes. Also, some of these are making our life easier.

┆Issue is synchronized with this Jira Story by Unito

[burmanm]

Also, we need to move away from the controller-runtime YAML config as it was removed in 0.18.0.