Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for svc backend #1363

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion api/v1beta1/gslb_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,9 @@ type Strategy struct {
// +k8s:openapi-gen=true
type GslbSpec struct {
// Gslb-enabled Ingress Spec
Ingress IngressSpec `json:"ingress"`
Ingress *IngressSpec `json:"ingress,omitempty"`
// Gslb-enabled LoadBalancer Service
LoadBalancer *LoadBalancer `json:"loadBalancer,omitempty"`
// Gslb Strategy spec
Strategy Strategy `json:"strategy"`
}
Expand Down
15 changes: 13 additions & 2 deletions api/v1beta1/upstream_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,17 @@ func (in *IngressSpec) DeepCopyInto(out *IngressSpec) {
}
}

// LoadBalancer defines the desired state of Gslb
// +k8s:openapi-gen=true
type LoadBalancer struct {
// Host is the fully qualified domain name of a network host, as defined by RFC 3986.
// This is the host name to be used as global domain name while creating GSLB records.
Host string `json:"host"`
// ServiceName is the name of the Service resource being exposed.
// Only LoadBalancer Service type is supported.
ServiceName string `json:"serviceName"`
}

// DeepCopyInto copying the receiver, writing into out. in must be non-nil.
func (in *IngressRule) DeepCopyInto(out *IngressRule) {
*out = *in
Expand All @@ -163,7 +174,7 @@ func (in *IngressRuleValue) DeepCopyInto(out *IngressRuleValue) {
}

// FromV1IngressSpec transforms from networking.k8s.io/v1 IngressSpec to custom k8gb ingress Spec
func FromV1IngressSpec(v1Spec netv1.IngressSpec) IngressSpec {
func FromV1IngressSpec(v1Spec netv1.IngressSpec) *IngressSpec {
spec := IngressSpec{}
spec.DefaultBackend = v1Spec.DefaultBackend
spec.IngressClassName = v1Spec.IngressClassName
Expand All @@ -177,7 +188,7 @@ func FromV1IngressSpec(v1Spec netv1.IngressSpec) IngressSpec {
}
spec.Rules = append(spec.Rules, rule)
}
return spec
return &spec
}

// ToV1IngressSpec transforms from k8gb ingress Spec to networking.k8s.io/v1 IngressSpec
Expand Down
25 changes: 24 additions & 1 deletion api/v1beta1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 16 additions & 1 deletion chart/k8gb/crd/k8gb.absa.oss_gslbs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,22 @@ spec:
type: object
type: array
type: object
loadBalancer:
description: Gslb-enabled LoadBalancer Service
properties:
host:
description: Host is the fully qualified domain name of a network
host, as defined by RFC 3986. This is the host name to be used
as global domain name while creating GSLB records.
type: string
serviceName:
description: ServiceName is the name of the Service resource being
exposed. Only LoadBalancer Service type is supported.
type: string
required:
- host
- serviceName
type: object
strategy:
description: Gslb Strategy spec
properties:
Expand All @@ -331,7 +347,6 @@ spec:
- type
type: object
required:
- ingress
- strategy
type: object
status:
Expand Down
16 changes: 13 additions & 3 deletions controllers/dnsupdate.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,19 @@ func (r *GslbReconciler) gslbDNSEndpoint(gslb *k8gbv1beta1.Gslb) (*externaldns.D
return nil, err
}

localTargets, err := r.DNSProvider.GslbIngressExposedIPs(gslb)
if err != nil {
return nil, err
localTargets := []string{}
if gslb.Spec.Ingress != nil {
localTargets, err = r.DNSProvider.GslbIngressExposedIPs(gslb)
if err != nil {
return nil, err
}
}

if gslb.Spec.LoadBalancer != nil {
localTargets, err = r.DNSProvider.GslbServiceExposedIPs(gslb)
Copy link
Member

@jkremser jkremser Dec 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this way, if both gslb.spec.loadBalancer and gslb.spec.ingress are specified in the CR, then the localTargets will be obtained using the LB section only (the last will win). I am not sure how it should be designed, to be honest. Does it actually make sense that gslb CR is backed by both at once? If not we could implement the x-kubernetes-validations to catch this during resource admission, otherwise I'd go with simple if - else if here for now.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I think adding a validation for it is better so that both can't be specified. It is a cleaner and better approach then making some blunders while missing some case in if - else approach. It is better to be clear at the start of the implementation and state that both can't be used at once. WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense to me. Would you mind exploring those x-kubernetes-validations? There should be an annotation for kubebuilder for that (link)

something like

// +kubebuilder:validation:XValidation:rule=( has(self.loadBalancer) && !has(self. ingress) ) || ( !has(self.loadBalancer) && has(self. ingress) )

if err != nil {
return nil, err
}
}

for host, health := range serviceHealth {
Expand Down
20 changes: 11 additions & 9 deletions controllers/gslb_controller_reconciliation.go
Original file line number Diff line number Diff line change
Expand Up @@ -137,16 +137,18 @@ func (r *GslbReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.
}

// == Ingress ==========
ingress, err := r.gslbIngress(gslb)
if err != nil {
m.IncrementError(gslb)
return result.RequeueError(err)
}
if gslb.Spec.Ingress != nil {
ingress, err := r.gslbIngress(gslb)
if err != nil {
m.IncrementError(gslb)
return result.RequeueError(err)
}

err = r.saveIngress(gslb, ingress)
if err != nil {
m.IncrementError(gslb)
return result.RequeueError(err)
err = r.saveIngress(gslb, ingress)
if err != nil {
m.IncrementError(gslb)
return result.RequeueError(err)
}
}

// == external-dns dnsendpoints CRs ==
Expand Down
13 changes: 9 additions & 4 deletions controllers/gslb_controller_setup.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,18 @@ func (r *GslbReconciler) SetupWithManager(mgr ctrl.Manager) error {
}
gslbName := ""
for _, gslb := range gslbList.Items {
for _, rule := range gslb.Spec.Ingress.Rules {
for _, path := range rule.HTTP.Paths {
if path.Backend.Service != nil && path.Backend.Service.Name == a.GetName() {
gslbName = gslb.Name
if gslb.Spec.Ingress != nil {
for _, rule := range gslb.Spec.Ingress.Rules {
for _, path := range rule.HTTP.Paths {
if path.Backend.Service != nil && path.Backend.Service.Name == a.GetName() {
gslbName = gslb.Name
}
}
}
}
if gslb.Spec.LoadBalancer != nil && gslb.Spec.LoadBalancer.ServiceName == a.GetName() {
gslbName = gslb.Name
}
}
if len(gslbName) > 0 {
return []reconcile.Request{
Expand Down
2 changes: 1 addition & 1 deletion controllers/ingress.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ func (r *GslbReconciler) gslbIngress(gslb *k8gbv1beta1.Gslb) (*netv1.Ingress, er
Namespace: gslb.Namespace,
Annotations: gslb.Annotations,
},
Spec: k8gbv1beta1.ToV1IngressSpec(gslb.Spec.Ingress),
Spec: k8gbv1beta1.ToV1IngressSpec(*gslb.Spec.Ingress),
}

err := controllerutil.SetControllerReference(gslb, ingress, r.Scheme)
Expand Down
15 changes: 15 additions & 0 deletions controllers/mocks/assistant_mock.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 15 additions & 0 deletions controllers/mocks/provider_mock.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions controllers/providers/assistant/assistant.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ type Assistant interface {
CoreDNSExposedIPs() ([]string, error)
// GslbIngressExposedIPs retrieves list of IP's exposed by all GSLB ingresses
GslbIngressExposedIPs(gslb *k8gbv1beta1.Gslb) ([]string, error)
// GslbServiceExposedIPs retrieves list of IP's exposed by all GSLB LoadBalancer type Services
GslbServiceExposedIPs(gslb *k8gbv1beta1.Gslb) ([]string, error)
// GetExternalTargets retrieves slice of targets from external clusters
GetExternalTargets(host string, extClusterNsNames map[string]string) (targets Targets)
// SaveDNSEndpoint update DNS endpoint or create new one if doesnt exist
Expand Down
41 changes: 41 additions & 0 deletions controllers/providers/assistant/gslb.go
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,47 @@ func (r *Gslb) GslbIngressExposedIPs(gslb *k8gbv1beta1.Gslb) ([]string, error) {
return gslbIngressIPs, nil
}

// GslbServiceExposedIPs retrieves list of IP's exposed by all GSLB LoadBalancer type services
func (r *Gslb) GslbServiceExposedIPs(gslb *k8gbv1beta1.Gslb) ([]string, error) {
nn := types.NamespacedName{
Name: gslb.Spec.LoadBalancer.ServiceName,
Namespace: gslb.Namespace,
}

gslbService := &corev1.Service{}

err := r.client.Get(context.TODO(), nn, gslbService)
if err != nil {
if errors.IsNotFound(err) {
log.Info().
Str("gslb", gslb.Spec.LoadBalancer.ServiceName).
Msg("Can't find gslb Service")
}
return nil, err
}

if gslbService.Spec.Type != corev1.ServiceTypeLoadBalancer {
return nil, fmt.Errorf("service %s is not of type LoadBalancer", gslb.Spec.LoadBalancer.ServiceName)
}

var gslbServiceIPs []string
for _, ip := range gslbService.Status.LoadBalancer.Ingress {
if len(ip.IP) > 0 {
gslbServiceIPs = append(gslbServiceIPs, ip.IP)
}
if len(ip.Hostname) > 0 {
IPs, err := utils.Dig(ip.Hostname, r.edgeDNSServers...)
if err != nil {
log.Warn().Err(err).Msg("Dig error")
return nil, err
}
gslbServiceIPs = append(gslbServiceIPs, IPs...)
}
}

return gslbServiceIPs, nil
}

// SaveDNSEndpoint update DNS endpoint or create new one if doesnt exist
func (r *Gslb) SaveDNSEndpoint(namespace string, i *externaldns.DNSEndpoint) error {
found := &externaldns.DNSEndpoint{}
Expand Down
2 changes: 2 additions & 0 deletions controllers/providers/dns/dns.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ type Provider interface {
CreateZoneDelegationForExternalDNS(*k8gbv1beta1.Gslb) error
// GslbIngressExposedIPs retrieves list of IP's exposed by all GSLB ingresses
GslbIngressExposedIPs(*k8gbv1beta1.Gslb) ([]string, error)
// GslbServiceExposedIPs retrieves list of IP's exposed by all LoadBalancer type Services
GslbServiceExposedIPs(*k8gbv1beta1.Gslb) ([]string, error)
// GetExternalTargets retrieves list of external targets for specified host
GetExternalTargets(string) assistant.Targets
// SaveDNSEndpoint update DNS endpoint in gslb or create new one if doesn't exist
Expand Down
4 changes: 4 additions & 0 deletions controllers/providers/dns/empty.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ func (p *EmptyDNSProvider) GslbIngressExposedIPs(gslb *k8gbv1beta1.Gslb) (r []st
return p.assistant.GslbIngressExposedIPs(gslb)
}

func (p *EmptyDNSProvider) GslbServiceExposedIPs(gslb *k8gbv1beta1.Gslb) (r []string, err error) {
return p.assistant.GslbServiceExposedIPs(gslb)
}

func (p *EmptyDNSProvider) GetExternalTargets(host string) (targets assistant.Targets) {
return p.assistant.GetExternalTargets(host, p.config.GetExternalClusterNSNames())
}
Expand Down
4 changes: 4 additions & 0 deletions controllers/providers/dns/external.go
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,10 @@ func (p *ExternalDNSProvider) GslbIngressExposedIPs(gslb *k8gbv1beta1.Gslb) ([]s
return p.assistant.GslbIngressExposedIPs(gslb)
}

func (p *ExternalDNSProvider) GslbServiceExposedIPs(gslb *k8gbv1beta1.Gslb) ([]string, error) {
return p.assistant.GslbServiceExposedIPs(gslb)
}

func (p *ExternalDNSProvider) SaveDNSEndpoint(gslb *k8gbv1beta1.Gslb, i *externaldns.DNSEndpoint) error {
return p.assistant.SaveDNSEndpoint(gslb.Namespace, i)
}
Expand Down
4 changes: 4 additions & 0 deletions controllers/providers/dns/infoblox.go
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,10 @@ func (p *InfobloxProvider) GslbIngressExposedIPs(gslb *k8gbv1beta1.Gslb) ([]stri
return p.assistant.GslbIngressExposedIPs(gslb)
}

func (p *InfobloxProvider) GslbServiceExposedIPs(gslb *k8gbv1beta1.Gslb) ([]string, error) {
return p.assistant.GslbServiceExposedIPs(gslb)
}

func (p *InfobloxProvider) SaveDNSEndpoint(gslb *k8gbv1beta1.Gslb, i *externaldns.DNSEndpoint) error {
return p.assistant.SaveDNSEndpoint(gslb.Namespace, i)
}
Expand Down
Loading
Loading