Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update grafana and prometheus to helm chart version #4542

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
3f882b3
feat: update grafana and prometheus version
moko-poi Aug 11, 2023
b847de8
feat: add loadListenerRules
moko-poi Aug 7, 2023
ffecec9
feat: add ModifyListenerRules
moko-poi Aug 7, 2023
a0cc81d
feat: add LoadListenerRules
moko-poi Aug 7, 2023
ee00e42
feat: add listener rules for ECSDeploymentInput
moko-poi Aug 7, 2023
2c4d7ef
feat: add loadListenerRules
moko-poi Aug 7, 2023
faee2fa
feat: add rules for ensureTrafficRouting
moko-poi Aug 7, 2023
1778624
feat: add modyfylistenerRules for routing
moko-poi Aug 7, 2023
de45786
feat: add ModifyListenerRules to ELB interface
moko-poi Aug 7, 2023
3a30dd9
feat: add listener_rules_test.go
moko-poi Aug 9, 2023
fbe5623
feat: add listenerrule to document
moko-poi Aug 9, 2023
f63d0f0
refactor: currListenerArns
moko-poi Aug 9, 2023
58e2755
reafactor: if-statement for listener rules to a more appropriate form
moko-poi Aug 10, 2023
36440ca
fix: listernerRules directly contains rules
moko-poi Aug 14, 2023
cd4f1a7
feat: Validate provided listener rules against existing rules
moko-poi Aug 18, 2023
cca0f6d
Merge pull request #1 from moko-poi/feat/selection-listener-rule-for-ecs
moko-poi Aug 23, 2023
b513b91
Merge branch 'pipe-cd:master' into master
moko-poi Aug 26, 2023
571c76a
Merge branch 'pipe-cd:master' into master
moko-poi Aug 28, 2023
6a9e107
Merge branch 'pipe-cd:master' into master
moko-poi Aug 28, 2023
b57f242
Merge branch 'pipe-cd:master' into master
moko-poi Sep 15, 2023
f9eda02
Merge branch 'pipe-cd:master' into master
moko-poi Oct 17, 2023
ee06d00
Merge branch 'master' into feat/update-grafana-and-prometheus-version
moko-poi Oct 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,7 @@ One of `yamlField` or `regex` is required.
| serviceDefinitionFile | string | The path ECS Service configuration file. Allow file in both `yaml` and `json` format. The default value is `service.json`. See [here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service_definition_parameters.html) for parameters.| No |
| taskDefinitionFile | string | The path to ECS TaskDefinition configuration file. Allow file in both `yaml` and `json` format. The default value is `taskdef.json`. See [here](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) for parameters. | No |
| targetGroups | [ECSTargetGroupInput](#ecstargetgroupinput) | The target groups configuration, will be used to routing traffic to created task sets. | Yes (if you want to perform progressive delivery) |
| listenerRules | []string | Specifies the listener rules used for registering the ECS task set if exists. If rules are not specified, traffic will be routed to the default listener. | No |

### ECSTargetGroupInput

Expand All @@ -454,7 +455,6 @@ One of `yamlField` or `regex` is required.
| canary | ECSTargetGroupObject | The CANARY target group, will be used to register the CANARY ECS task set if exist. It's required to enable PipeCD to perform the multi-stage deployment. | No |

Note: You can get examples for those object from [here](../../examples/#ecs-applications).

## ECSQuickSync

| Field | Type | Description | Required |
Expand Down
4 changes: 2 additions & 2 deletions manifests/pipecd/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ appVersion: 0.0.0
# If you want to update the version of any child charts, bump the version to an arbitrary one.
dependencies:
- name: prometheus
version: "v14.6.0"
version: "v23.3.0"
repository: "https://prometheus-community.github.io/helm-charts"
condition: monitoring.enabled
- name: grafana
version: "6.16.2"
version: "6.58.8"
repository: "https://grafana.github.io/helm-charts"
condition: monitoring.enabled
7 changes: 6 additions & 1 deletion pkg/app/piped/executor/ecs/deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,12 @@ func (e *deployExecutor) ensureTrafficRouting(ctx context.Context) model.StageSt
return model.StageStatus_STAGE_FAILURE
}

if !routing(ctx, &e.Input, e.platformProviderName, e.platformProviderCfg, *primary, *canary) {
rules, ok := loadListenerRules(&e.Input, e.appCfg, e.deploySource)
if !ok {
return model.StageStatus_STAGE_FAILURE
}

if !routing(ctx, &e.Input, e.platformProviderName, e.platformProviderCfg, *primary, *canary, rules) {
return model.StageStatus_STAGE_FAILURE
}
return model.StageStatus_STAGE_SUCCESS
Expand Down
53 changes: 50 additions & 3 deletions pkg/app/piped/executor/ecs/ecs.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,24 @@ func loadTargetGroups(in *executor.Input, appCfg *config.ECSApplicationSpec, ds
return primary, canary, true
}

func loadListenerRules(in *executor.Input, appCfg *config.ECSApplicationSpec, ds *deploysource.DeploySource) ([]string, bool) {
in.LogPersister.Infof("Loading listener rules config at the commit %s", ds.Revision)

rules, err := provider.LoadListenerRules(appCfg.Input.ListenerRules)
if err != nil && !errors.Is(err, provider.ErrNoListenerRule) {
in.LogPersister.Errorf("Failed to load ListenerRules (%v)", err)
return nil, false
}

if errors.Is(err, provider.ErrNoTargetGroup) {
in.LogPersister.Infof("No listener rules were set at commit %s", ds.Revision)
return nil, true
}

in.LogPersister.Infof("Successfully loaded the ECS listener rules at commit %s", ds.Revision)
return rules, true
}

func applyTaskDefinition(ctx context.Context, cli provider.Client, taskDefinition types.TaskDefinition) (*types.TaskDefinition, error) {
td, err := cli.RegisterTaskDefinition(ctx, taskDefinition)
if err != nil {
Expand Down Expand Up @@ -408,7 +426,7 @@ func clean(ctx context.Context, in *executor.Input, platformProviderName string,
return true
}

func routing(ctx context.Context, in *executor.Input, platformProviderName string, platformProviderCfg *config.PlatformProviderECSConfig, primaryTargetGroup types.LoadBalancer, canaryTargetGroup types.LoadBalancer) bool {
func routing(ctx context.Context, in *executor.Input, platformProviderName string, platformProviderCfg *config.PlatformProviderECSConfig, primaryTargetGroup types.LoadBalancer, canaryTargetGroup types.LoadBalancer, listenerRules []string) bool {
client, err := provider.DefaultRegistry().Client(platformProviderName, platformProviderCfg, in.Logger)
if err != nil {
in.LogPersister.Errorf("Unable to create ECS client for the provider %s: %v", platformProviderName, err)
Expand Down Expand Up @@ -446,10 +464,39 @@ func routing(ctx context.Context, in *executor.Input, platformProviderName strin
return false
}

if err := client.ModifyListeners(ctx, currListenerArns, routingTrafficCfg); err != nil {
in.LogPersister.Errorf("Failed to routing traffic to PRIMARY/CANARY variants: %v", err)
if len(listenerRules) == 0 {
if err := client.ModifyListeners(ctx, currListenerArns, routingTrafficCfg); err != nil {
in.LogPersister.Errorf("Failed to routing traffic to PRIMARY/CANARY variants: %v", err)
return false
}

return true
}

rules, err := client.GetListenerRules(ctx, currListenerArns)
if err != nil {
in.LogPersister.Errorf("Failed to retrieve listener rules: %v", err)
return false
}

// Validate if the provided listenerRules exist in the retrieved rules
for _, providedRule := range listenerRules {
ruleExists := false
for _, existingRule := range rules {
if providedRule == existingRule {
ruleExists = true
break
}
}
if !ruleExists {
in.LogPersister.Errorf("Provided listener rule %s does not exist", providedRule)
return false
}
}

if err := client.ModifyListenerRules(ctx, listenerRules, routingTrafficCfg); err != nil {
in.LogPersister.Errorf("Failed to routing traffic to PRIMARY/CANARY variants: %v", err)
return false
}
return true
}
62 changes: 62 additions & 0 deletions pkg/app/piped/platformprovider/ecs/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,30 @@ func (c *client) GetListenerArns(ctx context.Context, targetGroup types.LoadBala
return arns, nil
}

func (c *client) GetListenerRules(ctx context.Context, listenerArns []string) ([]string, error) {
var ruleArns []string

// 各リスナーのルールを取得
for _, listenerArn := range listenerArns {
input := &elasticloadbalancingv2.DescribeRulesInput{
ListenerArn: aws.String(listenerArn),
}
output, err := c.elbClient.DescribeRules(ctx, input)
if err != nil {
return nil, err
}
for _, rule := range output.Rules {
ruleArns = append(ruleArns, *rule.RuleArn)
}
}

if len(ruleArns) == 0 {
return nil, platformprovider.ErrNotFound
}

return ruleArns, nil
}

func (c *client) getLoadBalancerArn(ctx context.Context, targetGroupArn string) (string, error) {
input := &elasticloadbalancingv2.DescribeTargetGroupsInput{
TargetGroupArns: []string{targetGroupArn},
Expand Down Expand Up @@ -470,6 +494,44 @@ func (c *client) ModifyListeners(ctx context.Context, listenerArns []string, rou
return nil
}

func (c *client) ModifyListenerRules(ctx context.Context, listenerRuleArns []string, routingTrafficCfg RoutingTrafficConfig) error {
if len(routingTrafficCfg) != 2 {
return fmt.Errorf("invalid listener configuration: requires 2 target groups")
}

modifyListenerRule := func(ctx context.Context, listenerRuleArn string) error {
input := &elasticloadbalancingv2.ModifyRuleInput{
RuleArn: aws.String(listenerRuleArn),
Actions: []elbtypes.Action{
{
Type: elbtypes.ActionTypeEnumForward,
ForwardConfig: &elbtypes.ForwardActionConfig{
TargetGroups: []elbtypes.TargetGroupTuple{
{
TargetGroupArn: aws.String(routingTrafficCfg[0].TargetGroupArn),
Weight: aws.Int32(int32(routingTrafficCfg[0].Weight)),
},
{
TargetGroupArn: aws.String(routingTrafficCfg[1].TargetGroupArn),
Weight: aws.Int32(int32(routingTrafficCfg[1].Weight)),
},
},
},
},
},
}
_, err := c.elbClient.ModifyRule(ctx, input)
return err
}

for _, listener := range listenerRuleArns {
if err := modifyListenerRule(ctx, listener); err != nil {
return err
}
}
return nil
}

func (c *client) TagResource(ctx context.Context, resourceArn string, tags []types.Tag) error {
input := &ecs.TagResourceInput{
ResourceArn: aws.String(resourceArn),
Expand Down
7 changes: 7 additions & 0 deletions pkg/app/piped/platformprovider/ecs/ecs.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@ type ECS interface {

type ELB interface {
GetListenerArns(ctx context.Context, targetGroup types.LoadBalancer) ([]string, error)
GetListenerRules(ctx context.Context, listenerArns []string) ([]string, error)
ModifyListeners(ctx context.Context, listenerArns []string, routingTrafficCfg RoutingTrafficConfig) error
ModifyListenerRules(ctx context.Context, listenerRuleArns []string, routingTrafficCfg RoutingTrafficConfig) error
}

// Registry holds a pool of aws client wrappers.
Expand All @@ -82,6 +84,11 @@ func LoadTargetGroups(targetGroups config.ECSTargetGroups) (*types.LoadBalancer,
return loadTargetGroups(targetGroups)
}

// LoadListenerRules returns listener rules according to the defined in pipe definition file.
func LoadListenerRules(listenerRules []string) ([]string, error) {
return loadListenerRules(listenerRules)
}

type registry struct {
clients map[string]Client
mu sync.RWMutex
Expand Down
29 changes: 29 additions & 0 deletions pkg/app/piped/platformprovider/ecs/listener_rules.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// Copyright 2023 The PipeCD Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package ecs

import (
"errors"
)

var ErrNoListenerRule = errors.New("no target group")

func loadListenerRules(listenerRules []string) ([]string, error) {
if len(listenerRules) == 0 {
return nil, ErrNoListenerRule
}

return listenerRules, nil
}
61 changes: 61 additions & 0 deletions pkg/app/piped/platformprovider/ecs/listener_rules_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
// Copyright 2023 The PipeCD Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package ecs

import (
"testing"

"github.com/stretchr/testify/assert"
)

func TestLoadListenerRules(t *testing.T) {
t.Parallel()

testcases := []struct {
name string
listenerRules []string
wantRules []string
wantErr error
}{
{
name: "empty listener rules",
listenerRules: []string{},
wantRules: nil,
wantErr: ErrNoListenerRule,
},
{
name: "single listener rule",
listenerRules: []string{"rule1"},
wantRules: []string{"rule1"},
wantErr: nil,
},
{
name: "multiple listener rules",
listenerRules: []string{"rule1", "rule2", "rule3"},
wantRules: []string{"rule1", "rule2", "rule3"},
wantErr: nil,
},
}

for _, tc := range testcases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
rules, err := loadListenerRules(tc.listenerRules)
assert.Equal(t, tc.wantErr, err)
assert.Equal(t, tc.wantRules, rules)
})
}
}
2 changes: 2 additions & 0 deletions pkg/config/application_ecs.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ type ECSDeploymentInput struct {
TaskDefinitionFile string `json:"taskDefinitionFile" default:"taskdef.json"`
// ECSTargetGroups
TargetGroups ECSTargetGroups `json:"targetGroups"`
// ECSListenerRules
ListenerRules []string `json:"listenerRules"`
// Automatically reverts all changes from all stages when one of them failed.
// Default is true.
AutoRollback *bool `json:"autoRollback,omitempty" default:"true"`
Expand Down
Loading