Fix/metrics

[mneverov]

Requested metrics had existed, but were not exposed.

This patch:

• adds a new metrics server running on :9091 by default (see IPAM_METRICS_ADDR env. var)
• moves probe and metrics server to util
• replaces k8s.io/component-base/metrics/legacyregistry with prometheus/client_golang
• metrics are registered in an init func instead of using sync.Once

curl -v localhost:9091/metrics
# HELP node_ipam_controller_multicidrset_allocation_tries_per_request Histogram measuring CIDR allocation tries per request.
# TYPE node_ipam_controller_multicidrset_allocation_tries_per_request histogram
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="10.244.0.0/16",le="1"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="10.244.0.0/16",le="5"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="10.244.0.0/16",le="25"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="10.244.0.0/16",le="125"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="10.244.0.0/16",le="625"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="10.244.0.0/16",le="+Inf"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_sum{clusterCIDR="10.244.0.0/16"} 0
node_ipam_controller_multicidrset_allocation_tries_per_request_count{clusterCIDR="10.244.0.0/16"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="2001:db8::/110",le="1"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="2001:db8::/110",le="5"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="2001:db8::/110",le="25"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="2001:db8::/110",le="125"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="2001:db8::/110",le="625"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_bucket{clusterCIDR="2001:db8::/110",le="+Inf"} 3
node_ipam_controller_multicidrset_allocation_tries_per_request_sum{clusterCIDR="2001:db8::/110"} 0
node_ipam_controller_multicidrset_allocation_tries_per_request_count{clusterCIDR="2001:db8::/110"} 3
# HELP node_ipam_controller_multicidrset_cidrs_allocations_total Counter measuring total number of CIDR allocations.
# TYPE node_ipam_controller_multicidrset_cidrs_allocations_total counter
node_ipam_controller_multicidrset_cidrs_allocations_total{clusterCIDR="10.244.0.0/16"} 3
node_ipam_controller_multicidrset_cidrs_allocations_total{clusterCIDR="2001:db8::/110"} 3
# HELP node_ipam_controller_multicidrset_usage_cidrs Gauge measuring percentage of allocated CIDRs.
# TYPE node_ipam_controller_multicidrset_usage_cidrs gauge
node_ipam_controller_multicidrset_usage_cidrs{clusterCIDR="10.244.0.0/16"} 0.01171875
node_ipam_controller_multicidrset_usage_cidrs{clusterCIDR="2001:db8::/110"} 0.0029296875
# HELP node_ipam_controller_multicirdset_max_cidrs Maximum number of CIDRs that can be allocated.
# TYPE node_ipam_controller_multicirdset_max_cidrs gauge
node_ipam_controller_multicirdset_max_cidrs{clusterCIDR="10.244.0.0/16"} 256
node_ipam_controller_multicirdset_max_cidrs{clusterCIDR="2001:db8::/110"} 1024

Fixes #40

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mneverov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mneverov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mneverov]

@aojea ptal

[aojea]

• adds a new metrics server running on :9091 by default (see IPAM_METRICS_ADDR env. var)

we found that opening multiple ports for expose different endpoints ends causing problems, specially in host network pods, what do you think if we add a new flag --binding-address or similar to indicate the endpoint where the healthz, livez and metrics are exposed and mark the existing one health-probe-address as deprecated.

This way we just maintain one server and use one port, using the url to discriminate

[mneverov]

what do you think if ...

Good idea, wanted to combine those two initially, but then checked controller-runtime and it has 3 webservers: webhook, probe, and metrics.

[mneverov]

By default HealthProbeAddr is empty now. Add this check for existing clusters.
The default addr value :8081 from HealthProbeAddr is now specified for WebserverBindAddr

[aojea]

/lgtm

Thanks