Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add cloudian-sdk with CRUD on group #20

Merged
merged 52 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
47bc77d
feat: add cloudian-sdk with CRUD on group
mariatsji Nov 21, 2024
57cf3f6
refactor: reuse http.Client in sdk.go
mariatsji Nov 21, 2024
6c876a6
format: gofmt -s sdk.go
mariatsji Nov 21, 2024
f4632df
lint: remove decode-method, trust user
mariatsji Nov 21, 2024
f815199
lint: cloudian sdk inline status code check
mariatsji Nov 21, 2024
05284af
lint: cloudian sdk insert space after comment slashes
mariatsji Nov 21, 2024
c80b2dc
lint: cloudian sdk close all response bodys
mariatsji Nov 21, 2024
774bd09
lint: cloudian sdk annotate json structs
mariatsji Nov 21, 2024
ee23dca
refactor: cloudian sdk refer to http package verbs
mariatsji Nov 21, 2024
e130ccb
lint: cloudian sdk add explicit http context
mariatsji Nov 21, 2024
ea5cd14
refactor: adhere golang convension of capitalized abbreviations
mariatsji Nov 22, 2024
982451d
fix: add cloudian base url required env variable
mariatsji Nov 22, 2024
171e4f6
refactor: sdk cloudian, shorten offset var assignment
mariatsji Nov 22, 2024
fbfdbf5
refactor: cloudian sdk defer body close and ignore any failure doing so
mariatsji Nov 22, 2024
36bc3ba
fix: provider-cloudian fix delete user logic failure
mariatsji Nov 22, 2024
3a36ae8
Update internal/sdk/cloudian/sdk.go
mariatsji Nov 22, 2024
9ab98f6
cloudian sdk ignore error check of body close
mariatsji Nov 22, 2024
074b7f8
refactor: cloudian sdk strip away return types for non-fetching opera…
mariatsji Nov 22, 2024
768565d
refactor: introduce cludian sdk client
mariatsji Nov 22, 2024
a8f26e2
fix: cloudian sdk should not log
mariatsji Nov 22, 2024
52cdc12
refactor: cloudian sdk common headers function
mariatsji Nov 22, 2024
ddd5b78
test: add property test for json marshall roundtrip just for fun
mariatsji Nov 22, 2024
1c25a64
Update internal/sdk/cloudian/sdk.go
mariatsji Nov 22, 2024
b93b115
Update internal/sdk/cloudian/sdk.go
mariatsji Nov 22, 2024
fbf08f7
Update internal/sdk/cloudian/sdk.go
mariatsji Nov 22, 2024
e8db770
fix: compilation errors after updates
mariatsji Nov 22, 2024
5c7ca3f
refactor: cloudian sdk centralize request creation
mariatsji Nov 22, 2024
17a8b27
fix: cludian sdk delete user error logic
mariatsji Nov 22, 2024
27177d0
fix: cloudian sdk errors
mariatsji Nov 22, 2024
15972b5
refactor: cloudian sdk allow callsite to pass context
mariatsji Nov 25, 2024
be4322a
fix: cloudian sdk no need to check ctx
mariatsji Nov 25, 2024
2d6abbd
refactor: cloudian sdk move util functions to the bottom
mariatsji Nov 25, 2024
8a908c9
refactor: cloudian sdk gofmt
mariatsji Nov 25, 2024
63adced
refactor: cloudian sdk no need to else here
mariatsji Nov 25, 2024
3855f66
refactor: cloudian sdk no need to else here
mariatsji Nov 25, 2024
d1b09cb
refactor: cloudian sdk rewrite getgroup logic to switch on resp.statu…
mariatsji Nov 25, 2024
4ca796e
refactor: cloudian sdk gofmt
mariatsji Nov 25, 2024
f33b22d
refactor: consistent use of error wrapping and defer body close
mariatsji Nov 25, 2024
0996284
doc: cloudian sdk minimal golang doc
mariatsji Nov 25, 2024
7dc35de
refactor: Initialisms ref go.dev wiki
mariatsji Nov 25, 2024
65a875e
test: fix cloudian sdk test with utf-8 chars
mariatsji Nov 25, 2024
48d1baa
refactor: cloudian sdk, do not export http client
mariatsji Nov 25, 2024
7dff035
fix: cloudian sdk casing fix
mariatsji Nov 25, 2024
6161a7b
Update internal/sdk/cloudian/sdk.go
mariatsji Nov 25, 2024
ce7ece4
chore: move errorcheck closer to soursce
mariatsji Nov 25, 2024
9944c4b
chore: newline
mariatsji Nov 25, 2024
d586aa4
refactor: remove redundant marshalling util functions
mariatsji Nov 25, 2024
0b89ad8
refactor: cloudian sdk introduce custom GroupNotFoundError
mariatsji Nov 25, 2024
97aef55
feat: Add exposed ResponseReason enum and util method to check it
mariatsji Nov 26, 2024
2673e9b
refactor: cloudian sdk expose ErrNotFound to callsite
mariatsji Nov 26, 2024
a5ae747
fix: cloudian sdk pair programming fixes
mariatsji Nov 26, 2024
fd49cd1
fix: lint
mariatsji Nov 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
291 changes: 291 additions & 0 deletions internal/sdk/cloudian/sdk.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,291 @@
package cloudian

import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"time"
)

type Client struct {
baseURL string
httpClient *http.Client
token string
}

func MkClient(baseUrl string, tokenBase64 string) *Client {
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
return &Client{
baseURL: baseUrl,
httpClient: &http.Client{},
token: tokenBase64,
}
}

func (client Client) headerModifier(req *http.Request) {
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Basic "+client.token)
}

type Group struct {
Active string `json:"active"`
GroupID string `json:"groupId"`
GroupName string `json:"groupName"`
LDAPEnabled bool `json:"ldapEnabled"`
LDAPGroup string `json:"ldapGroup"`
LDAPMatchAttribute string `json:"ldapMatchAttribute"`
LDAPSearch string `json:"ldapSearch"`
LDAPSearchUserBase string `json:"ldapSearchUserBase"`
LDAPServerURL string `json:"ldapServerURL"`
LDAPUserDNTemplate string `json:"ldapUserDNTemplate"`
S3endpointshttp []string `json:"s3endpointshttp"`
S3endpointshttps []string `json:"s3endpointshttps"`
S3websiteendpoints []string `json:"s3websiteendpoints"`
}

type User struct {
UserID string `json:"userId"`
GroupID string `json:"groupId"`
CanonicalUserID string `json:"canonicalUserId"`
}

func marshalGroup(group Group) ([]byte, error) {
return json.Marshal(group)
}

func unmarshalGroupJson(data []byte) (Group, error) {
var group Group
err := json.Unmarshal(data, &group)
return group, err
}

func unmarshalUsersJson(data []byte) ([]User, error) {
var users []User
err := json.Unmarshal(data, &users)
return users, err
}
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

@erikgb erikgb Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these helpers add value? I think not, but they should be put at the bottom of the file. Ref. Go programming best practices.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to the bottom.. they are useful for tests only

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

helper-functions now removed!


// List all users of a group
func (client Client) ListUsers(groupId string, offsetUserId *string) ([]User, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should have Context as first param.

var retVal []User

limit := 100

var offsetQueryParam = ""
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
if offsetUserId != nil {
offsetQueryParam = "&offset=" + *offsetUserId
}

url := client.baseURL + "/user/list?groupId=" + groupId + "&userType=all&userStatus=all&limit=" + strconv.Itoa(limit) + offsetQueryParam

// Create a context with a timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, fmt.Errorf("GET error creating list request: %w", err)
}

resp, err := client.httpClient.Do(req)

if err == nil {
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
body, err := io.ReadAll(resp.Body)
defer resp.Body.Close() // nolint:errcheck
if err != nil {
return nil, fmt.Errorf("GET reading list users response body failed: %w", err)
}

users, err := unmarshalUsersJson(body)
if err != nil {
return nil, fmt.Errorf("GET unmarshal users response body failed: %w", err)
}

retVal = append(retVal, users...)

// list users is a paginated API endpoint, so we need to check the limit and use an offset to fetch more
if len(users) > limit {
// There is some ambiguity in the GET /user/list endpoint documentation, but it seems
// that UserId is the correct key for this parameter (and not CanonicalUserId)
// Fetch more results
moreUsers, err := client.ListUsers(groupId, &users[limit].UserID)

if err == nil {
retVal = append(retVal, moreUsers...)
}
}

return retVal, nil
} else {
return nil, fmt.Errorf("GET list users failed: %w", err)
}

}

// Delete a single user
func (client Client) DeleteUser(user User) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should have Context as first param.

url := client.baseURL + "/user?userId=" + user.UserID + "&groupId=" + user.GroupID + "&canonicalUserId=" + user.CanonicalUserID

// Create a context with a timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, url, nil)
if err != nil {
return fmt.Errorf("DELETE error creating request: %w", err)
}

client.headerModifier(req)

resp, err := client.httpClient.Do(req)

if resp != nil && err == nil {
// Cloudian does not return a payload for this DELETE, but we can echo it to the callsite if all went well
defer resp.Body.Close() // nolint:errcheck

return nil
}
return err
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
}

// Delete a group and all its members
func (client Client) DeleteGroupRecursive(groupId string) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should have Context as first param.

users, err := client.ListUsers(groupId, nil)

if err != nil {
for _, user := range users {
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
err := client.DeleteUser(user)
if err != nil {
return fmt.Errorf("Error deleting user: %w", err)
}
}

return client.DeleteGroup(groupId)
}

return err
}

// Deletes a group if it is without members
func (client Client) DeleteGroup(groupId string) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should have Context as first param.

url := client.baseURL + "/group?groupId=" + groupId

// Create a context with a timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, url, nil)
if err != nil {
return fmt.Errorf("DELETE error creating request: %w", err)
}

client.headerModifier(req)

resp, err := client.httpClient.Do(req)

if err != nil {
statusErrStr := strconv.Itoa(resp.StatusCode)
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
return fmt.Errorf("DELETE to cloudian /group got status code [%s]: %w", statusErrStr, err)
}
defer resp.Body.Close() // nolint:errcheck

return nil
}

func (client Client) CreateGroup(group Group) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should have Context as first param.

url := client.baseURL + "/group"

jsonData, err := marshalGroup(group)
if err != nil {
return fmt.Errorf("Error marshaling JSON: %w", err)
}

// Create a context with a timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewBuffer(jsonData))
if err != nil {
return fmt.Errorf("POST error creating request: %w", err)
}

client.headerModifier(req)

resp, err := client.httpClient.Do(req)

if err != nil {
statusErrStr := strconv.FormatInt(int64(resp.StatusCode), 10)
return fmt.Errorf("POST to cloudian /group got status code [%s]: %w", statusErrStr, err)
}
defer resp.Body.Close() // nolint:errcheck

return err
}

func (client Client) UpdateGroup(group Group) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should have Context as first param.

url := client.baseURL + "/group"

jsonData, err := marshalGroup(group)
if err != nil {
return fmt.Errorf("Error marshaling JSON: %w", err)
}

// Create a context with a timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodPut, url, bytes.NewBuffer(jsonData))
if err != nil {
return fmt.Errorf("PUT error creating request: %w", err)
}

client.headerModifier(req)

resp, err := client.httpClient.Do(req)

if err != nil {
statusErrStr := strconv.FormatInt(int64(resp.StatusCode), 10)
return fmt.Errorf("PUT to cloudian /group got status code [%s]: %w", statusErrStr, err)
}

defer resp.Body.Close() // nolint:errcheck

return nil
}

func (client Client) GetGroup(groupId string) (*Group, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function should have Context as first param.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, fixed

url := client.baseURL + "/group?groupId=" + groupId

// Create a context with a timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, fmt.Errorf("GET error creating request: %w", err)
}

client.headerModifier(req)

resp, err := client.httpClient.Do(req)

if err != nil {
return nil, fmt.Errorf("GET error: %w", err)
}

defer resp.Body.Close() // nolint:errcheck

if resp.StatusCode == 200 {
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("GET reading response body failed: %w", err)
}

group, err := unmarshalGroupJson(body)
if err != nil {
return nil, fmt.Errorf("GET unmarshal response body failed: %w", err)
}

return &group, nil
}

// Cloudian-API returns 204 if the group does not exist
return nil, nil
mariatsji marked this conversation as resolved.
Show resolved Hide resolved
}
Loading
Loading