generated from openmcp-project/repository-template
-
Notifications
You must be signed in to change notification settings - Fork 2
feat: add utility functions for scheduler #66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Diaphteiros
wants to merge
3
commits into
main
Choose a base branch
from
scheduler
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,186
−39
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
# Retrying k8s Operations | ||
|
||
The `pkg/retry` package contains a `Client` that wraps a `client.Client` while implementing the interface itself and retries any failed (= the returned error is not `nil`) operation. | ||
Methods that don't return an error are simply forwarded to the internal client. | ||
|
||
In addition to the `client.Client` interface's methods, the `retry.Client` also has `CreateOrUpdate` and `CreateOrPatch` methods, which use the corresponding controller-runtime implementations internally. | ||
|
||
The default retry parameters are: | ||
- retry every 100 milliseconds | ||
- don't increase retry interval | ||
- no maximum number of attempts | ||
- timeout after 1 second | ||
|
||
The `retry.Client` struct has builder-style methods to configure the parameters: | ||
```golang | ||
retryingClient := retry.NewRetryingClient(myClient). | ||
WithTimeout(10 * time.Second). // try for at max 10 seconds | ||
WithInterval(500 * time.Millisecond). // try every 500 milliseconds, but ... | ||
WithBackoffMultiplier(2.0) // ... double the interval after each retry | ||
``` | ||
|
||
For convenience, the `clusters.Cluster` type can return a retrying client for its internal client: | ||
```golang | ||
// cluster is of type *clusters.Cluster | ||
err := cluster.Retry().WithMaxAttempts(3).Get(...) | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,11 @@ | ||
package maps | ||
|
||
import "github.com/openmcp-project/controller-utils/pkg/collections/filters" | ||
import ( | ||
"k8s.io/utils/ptr" | ||
|
||
"github.com/openmcp-project/controller-utils/pkg/collections/filters" | ||
"github.com/openmcp-project/controller-utils/pkg/pairs" | ||
) | ||
|
||
// Filter filters a map by applying a filter function to each key-value pair. | ||
// Only the entries for which the filter function returns true are kept in the copy. | ||
|
@@ -50,3 +55,34 @@ func Intersect[K comparable, V any](source map[K]V, maps ...map[K]V) map[K]V { | |
|
||
return res | ||
} | ||
|
||
// MapKeys returns a slice of all keys in the map. | ||
// The order is unspecified. | ||
// The keys are not deep-copied, so changes to them could affect the original map. | ||
func MapKeys[K comparable, V any](m map[K]V) []K { | ||
keys := make([]K, 0, len(m)) | ||
for k := range m { | ||
keys = append(keys, k) | ||
} | ||
return keys | ||
} | ||
|
||
// MapValues returns a slice of all values in the map. | ||
// The order is unspecified. | ||
// The values are not deep-copied, so changes to them could affect the original map. | ||
func MapValues[K comparable, V any](m map[K]V) []V { | ||
values := make([]V, 0, len(m)) | ||
for _, v := range m { | ||
values = append(values, v) | ||
} | ||
return values | ||
} | ||
Comment on lines
+70
to
+79
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The same goes here with https://pkg.go.dev/maps#Values as well. What do you think about that? |
||
|
||
// GetAny returns an arbitrary key-value pair from the map as a pointer to a pairs.Pair. | ||
// If the map is empty, it returns nil. | ||
func GetAny[K comparable, V any](m map[K]V) *pairs.Pair[K, V] { | ||
for k, v := range m { | ||
return ptr.To(pairs.New(k, v)) | ||
} | ||
return nil | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
package collections | ||
|
||
// ProjectSlice takes a slice and a projection function and applies this function to each element of the slice. | ||
// It returns a new slice containing the results of the projection. | ||
// The original slice is not modified. | ||
// If the projection function is nil, it returns nil. | ||
func ProjectSlice[X any, Y any](src []X, project func(X) Y) []Y { | ||
if project == nil { | ||
return nil | ||
} | ||
res := make([]Y, len(src)) | ||
for i, src := range src { | ||
res[i] = project(src) | ||
} | ||
return res | ||
} | ||
|
||
// ProjectMapToSlice takes a map and a projection function and applies this function to each key-value pair in the map. | ||
// It returns a new slice containing the results of the projection. | ||
// The original map is not modified. | ||
// If the projection function is nil, it returns nil. | ||
func ProjectMapToSlice[K comparable, V any, R any](src map[K]V, project func(K, V) R) []R { | ||
if project == nil { | ||
return nil | ||
} | ||
res := make([]R, 0, len(src)) | ||
for k, v := range src { | ||
res = append(res, project(k, v)) | ||
} | ||
return res | ||
} | ||
|
||
// ProjectMapToMap takes a map and a projection function and applies this function to each key-value pair in the map. | ||
// It returns a new map containing the results of the projection. | ||
// The original map is not modified. | ||
// Note that the resulting map may be smaller if the projection function does not guarantee unique keys. | ||
// If the projection function is nil, it returns nil. | ||
func ProjectMapToMap[K1 comparable, V1 any, K2 comparable, V2 any](src map[K1]V1, project func(K1, V1) (K2, V2)) map[K2]V2 { | ||
if project == nil { | ||
return nil | ||
} | ||
res := make(map[K2]V2, len(src)) | ||
for k, v := range src { | ||
newK, newV := project(k, v) | ||
res[newK] = newV | ||
} | ||
return res | ||
} | ||
|
||
// AggregateSlice takes a slice, an aggregation function and an initial value. | ||
// It applies the aggregation function to each element of the slice, also passing in the current result. | ||
// For the first element, it uses the initial value as the current result. | ||
// Returns initial if the aggregation function is nil. | ||
func AggregateSlice[X any, Y any](src []X, agg func(X, Y) Y, initial Y) Y { | ||
if agg == nil { | ||
return initial | ||
} | ||
res := initial | ||
for _, x := range src { | ||
res = agg(x, res) | ||
} | ||
return res | ||
} | ||
|
||
// AggregateMap takes a map, an aggregation function and an initial value. | ||
// It applies the aggregation function to each key-value pair in the map, also passing in the current result. | ||
// For the first key-value pair, it uses the initial value as the current result. | ||
// Returns initial if the aggregation function is nil. | ||
func AggregateMap[K comparable, V any, R any](src map[K]V, agg func(K, V, R) R, initial R) R { | ||
if agg == nil { | ||
return initial | ||
} | ||
res := initial | ||
for k, v := range src { | ||
res = agg(k, v, res) | ||
} | ||
return res | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
package collections_test | ||
|
||
import ( | ||
. "github.com/onsi/ginkgo/v2" | ||
. "github.com/onsi/gomega" | ||
|
||
"github.com/openmcp-project/controller-utils/pkg/collections" | ||
) | ||
|
||
var _ = Describe("Utils Tests", func() { | ||
|
||
Context("ProjectSlice", func() { | ||
|
||
projectFunc := func(i int) int { | ||
return i * 2 | ||
} | ||
|
||
It("should use the projection function on each element of the slice", func() { | ||
src := []int{1, 2, 3, 4} | ||
projected := collections.ProjectSlice(src, projectFunc) | ||
Expect(projected).To(Equal([]int{2, 4, 6, 8})) | ||
Expect(src).To(Equal([]int{1, 2, 3, 4}), "original slice should not be modified") | ||
}) | ||
|
||
It("should return an empty slice for an empty or nil input slice", func() { | ||
Expect(collections.ProjectSlice(nil, projectFunc)).To(BeEmpty()) | ||
Expect(collections.ProjectSlice([]int{}, projectFunc)).To(BeEmpty()) | ||
}) | ||
|
||
It("should return nil for a nil projection function", func() { | ||
src := []int{1, 2, 3, 4} | ||
projected := collections.ProjectSlice[int, int](src, nil) | ||
Expect(projected).To(BeNil()) | ||
Expect(src).To(Equal([]int{1, 2, 3, 4}), "original slice should not be modified") | ||
}) | ||
|
||
}) | ||
|
||
Context("ProjectMapToSlice", func() { | ||
|
||
projectFunc := func(k string, v string) string { | ||
return k + ":" + v | ||
} | ||
|
||
It("should use the projection function on each key-value pair of the map", func() { | ||
src := map[string]string{"a": "1", "b": "2", "c": "3"} | ||
projected := collections.ProjectMapToSlice(src, projectFunc) | ||
Expect(projected).To(ConsistOf("a:1", "b:2", "c:3")) | ||
Expect(src).To(Equal(map[string]string{"a": "1", "b": "2", "c": "3"}), "original map should not be modified") | ||
}) | ||
|
||
It("should return an empty slice for an empty or nil input map", func() { | ||
Expect(collections.ProjectMapToSlice(nil, projectFunc)).To(BeEmpty()) | ||
Expect(collections.ProjectMapToSlice(map[string]string{}, projectFunc)).To(BeEmpty()) | ||
}) | ||
|
||
It("should return nil for a nil projection function", func() { | ||
src := map[string]string{"a": "1", "b": "2", "c": "3"} | ||
projected := collections.ProjectMapToSlice[string, string, string](src, nil) | ||
Expect(projected).To(BeNil()) | ||
Expect(src).To(Equal(map[string]string{"a": "1", "b": "2", "c": "3"}), "original map should not be modified") | ||
}) | ||
|
||
}) | ||
|
||
Context("ProjectMapToMap", func() { | ||
|
||
projectFunc := func(k string, v string) (string, int) { | ||
return k, len(v) | ||
} | ||
|
||
It("should use the projection function on each key-value pair of the map", func() { | ||
src := map[string]string{"a": "1", "b": "22", "c": "333"} | ||
projected := collections.ProjectMapToMap(src, projectFunc) | ||
Expect(projected).To(Equal(map[string]int{"a": 1, "b": 2, "c": 3})) | ||
Expect(src).To(Equal(map[string]string{"a": "1", "b": "22", "c": "333"}), "original map should not be modified") | ||
}) | ||
|
||
It("should return an empty map for an empty or nil input map", func() { | ||
Expect(collections.ProjectMapToMap(nil, projectFunc)).To(BeEmpty()) | ||
Expect(collections.ProjectMapToMap(map[string]string{}, projectFunc)).To(BeEmpty()) | ||
}) | ||
|
||
It("should return nil for a nil projection function", func() { | ||
src := map[string]string{"a": "1", "b": "22", "c": "333"} | ||
projected := collections.ProjectMapToMap[string, string, string, int](src, nil) | ||
Expect(projected).To(BeNil()) | ||
Expect(src).To(Equal(map[string]string{"a": "1", "b": "22", "c": "333"}), "original map should not be modified") | ||
}) | ||
|
||
}) | ||
|
||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we instead use the
Keys
function from the standard package here as well: https://pkg.go.dev/maps#Keys? Or does this not work, with what you have planned?