Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New service for centralized config profile #1002

Merged
merged 17 commits into from
Aug 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1dadf5b
Added new ConfigProfile service with tests
eranturgeman Aug 14, 2024
4c36382
Added a question to be removed or resolved before merge
eranturgeman Aug 14, 2024
6bcd0f0
Deleted questions and applied mock servers for tests
eranturgeman Aug 14, 2024
7c3b8e3
Merge branch 'dev' into centralized-config-implementation
eranturgeman Aug 15, 2024
5046fb6
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into c…
eranturgeman Aug 15, 2024
80dd812
updated README.md
eranturgeman Aug 15, 2024
e8a94fe
Merge remote-tracking branch 'erant-fork/centralized-config-implement…
eranturgeman Aug 15, 2024
7097ba1
Fixed CR notes
eranturgeman Aug 18, 2024
667517a
Merge branch 'dev' into centralized-config-implementation
eranturgeman Aug 19, 2024
5025e90
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into c…
eranturgeman Aug 19, 2024
e887308
Merge remote-tracking branch 'erant-fork/centralized-config-implement…
eranturgeman Aug 19, 2024
2b511c0
removed documentation for new Get Config Profile service
eranturgeman Aug 20, 2024
b77673d
Merge branch 'dev' of https://github.com/jfrog/jfrog-client-go into c…
eranturgeman Aug 21, 2024
bf5dbce
Upgrading ConfigProfileMinXscVersion to actual minimal version requir…
eranturgeman Aug 25, 2024
1873b22
checking if the failure in tests happens because of the version upgrade
eranturgeman Aug 25, 2024
b3bbb14
updating minimal xsc minimal version required for centralized config
eranturgeman Aug 25, 2024
135265b
Merge branch 'dev' into centralized-config-implementation
eyalbe4 Aug 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,9 +184,9 @@
- [Using XSC Services](#using-xsc-services)
- [Fetching XSC's Version](#fetching-xscs-version)
- [Report XSC analytics metrics](#report-xsc-analytics-metrics)
- [Add analytics general event](#add-analytics-general-event)
- [Update analytics general event](#update-analytics-general-event)
- [Get analytics general event](#get-analytics-general-event)
- [Add Analytics General Event](#add-analytics-general-event)
- [Update Analytics General Event](#update-analytics-general-event)
- [Get Analytics General Event](#get-analytics-general-event)
- [Pipelines APIs](#pipelines-apis)
- [Creating Pipelines Service Manager](#creating-pipelines-service-manager)
- [Creating Pipelines Details](#creating-pipelines-details)
Expand Down Expand Up @@ -2468,8 +2468,8 @@ xscManager, err := xsc.New(serviceConfig)
version, err := xscManager.GetVersion()
```

#### Report XSC analytics metrics
##### Add analytics general event
#### Report XSC Analytics Metrics
##### Add Analytics General Event
Sent XSC a new event which contains analytics data, and get multi-scan id back from XSC.
```go
event := services.XscAnalyticsGeneralEvent{
Expand All @@ -2486,7 +2486,7 @@ event := services.XscAnalyticsGeneralEvent{
}}
msi, err := xscManager.AddAnalyticsGeneralEvent(event)
```
##### Update analytics general event
##### Update Analytics General Event
Sent XSC a finalized analytics metrics event with information matching an existing event's msi.
```go
finalizeEvent := services.XscAnalyticsGeneralEventFinalize{
Expand All @@ -2501,7 +2501,7 @@ finalizeEvent := services.XscAnalyticsGeneralEventFinalize{
err := xscManager.UpdateAnalyticsGeneralEvent(finalizeEvent)
```

##### Get analytics general event
##### Get Analytics General Event
Get a general event from XSC matching the provided msi.
```go
event, err := xscManager.GetAnalyticsGeneralEvent(msi)
Expand Down
49 changes: 49 additions & 0 deletions tests/testdata/configprofile/configProfileExample.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"profile_name": "default-profile",
"frogbot_config": {
"email_author": "[email protected]",
"aggregate_fixes": true,
"avoid_previous_pr_comments_deletion": true,
"branch_name_template": "frogbot-${IMPACTED_PACKAGE}-${BRANCH_NAME_HASH}",
"pr_title_template": "[🐸 Frogbot] Upgrade {IMPACTED_PACKAGE} to {FIX_VERSION}",
"pr_comment_title": "Frogbot notes:",
"commit_message_template": "Upgrade {IMPACTED_PACKAGE} to {FIX_VERSION}",
"show_secrets_as_pr_comment": false
},
"modules": [
{
"module_name": "default-module",
"path_from_root": ".",
"releases_repo": "nuget-remote",
"analyzer_manager_version": "1.8.1",
"additional_paths_for_module": ["lib1", "utils/lib2"],
"exclude_paths": ["**/.git/**", "**/*test*/**", "**/*venv*/**", "**/*node_modules*/**", "**/target/**"],
"scan_config": {
"scan_timeout": 600,
"exclude_pattern": "*.md",
"enable_sca_scan": true,
"enable_contextual_analysis_scan": true,
"sast_scanner_config": {
"enable_sast_scan": true
},
"secrets_scanner_config": {
"enable_secrets_scan": true
},
"iac_scanner_config": {
"enable_iac_scan": true
},
"applications_scanner_config": {
"enable_applications_scan": true
},
"services_scanner_config": {
"enable_services_scan": true
}
},
"protected_branches": ["main", "master"],
"include_exclude_mode": 0,
"include_exclude_pattern": "*test*",
"report_analytics": true
}
],
"is_default": true
}
54 changes: 54 additions & 0 deletions tests/xscconfigprofile_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
package tests

import (
"encoding/json"
"github.com/jfrog/jfrog-client-go/http/jfroghttpclient"
"github.com/jfrog/jfrog-client-go/xsc/services"
"github.com/stretchr/testify/assert"
"net/http"
"net/http/httptest"
"os"
"testing"
)

func TestGetConfigurationProfile(t *testing.T) {
initXscTest(t, services.ConfigProfileMinXscVersion)

mockServer, configProfileService := createXscMockServerForConfigProfile(t)
defer mockServer.Close()

configProfile, err := configProfileService.GetConfigurationProfile("default-test-profile")
assert.NoError(t, err)

profileFileContent, err := os.ReadFile("testdata/configprofile/configProfileExample.json")
assert.NoError(t, err)
var configProfileForComparison services.ConfigProfile
err = json.Unmarshal(profileFileContent, &configProfileForComparison)
assert.NoError(t, err)
assert.Equal(t, &configProfileForComparison, configProfile)
}

func createXscMockServerForConfigProfile(t *testing.T) (mockServer *httptest.Server, configProfileService *services.ConfigurationProfileService) {
mockServer = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.RequestURI == "/xsc/api/v1/profile/default-test-profile" && r.Method == http.MethodGet {
w.WriteHeader(http.StatusOK)
content, err := os.ReadFile("testdata/configprofile/configProfileExample.json")
assert.NoError(t, err)
_, err = w.Write(content)
assert.NoError(t, err)
} else {
assert.Fail(t, "received an unexpected request")
}
}))

xscDetails := GetXscDetails()
xscDetails.SetUrl(mockServer.URL + "/xsc")
xscDetails.SetAccessToken("")

client, err := jfroghttpclient.JfrogClientBuilder().Build()
assert.NoError(t, err)

configProfileService = services.NewConfigurationProfileService(client)
configProfileService.XscDetails = xscDetails
return
}
7 changes: 4 additions & 3 deletions tests/xsclogerrorevent_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ const errorMessageContentForTest = "THIS IS NOT A REAL ERROR! This Error is post

func TestXscSendLogErrorEvent(t *testing.T) {
initXscTest(t, services.LogErrorMinXscVersion)
mockServer, logErrorService := createXscMockServer(t)
mockServer, logErrorService := createXscMockServerForLogEvent(t)
defer mockServer.Close()

event := &services.ExternalErrorLog{
Expand All @@ -26,7 +26,7 @@ func TestXscSendLogErrorEvent(t *testing.T) {
assert.NoError(t, logErrorService.SendLogErrorEvent(event))
}

func createXscMockServer(t *testing.T) (mockServer *httptest.Server, logErrorService *services.LogErrorEventService) {
func createXscMockServerForLogEvent(t *testing.T) (mockServer *httptest.Server, logErrorService *services.LogErrorEventService) {
mockServer = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.RequestURI == "/xsc/api/v1/event/logMessage" && r.Method == http.MethodPost {
var reqBody services.ExternalErrorLog
Expand All @@ -42,8 +42,9 @@ func createXscMockServer(t *testing.T) (mockServer *httptest.Server, logErrorSer
assert.Equal(t, errorMessageContentForTest, reqBody.Message)
w.WriteHeader(http.StatusCreated)
return
} else {
assert.Fail(t, "received an unexpected request")
}
assert.Fail(t, "received an unexpected request")
}))

xscDetails := GetXscDetails()
Expand Down
6 changes: 6 additions & 0 deletions xsc/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,3 +75,9 @@ func (sm *XscServicesManager) GetAnalyticsGeneralEvent(msi string) (*services.Xs
eventService.XscDetails = sm.config.GetServiceDetails()
return eventService.GetGeneralEvent(msi)
}

func (sm *XscServicesManager) GetConfigProfile(profileName string) (*services.ConfigProfile, error) {
configProfileService := services.NewConfigurationProfileService(sm.client)
configProfileService.XscDetails = sm.config.GetServiceDetails()
return configProfileService.GetConfigurationProfile(profileName)
}
113 changes: 113 additions & 0 deletions xsc/services/profile.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
package services

import (
"encoding/json"
"fmt"
"github.com/jfrog/jfrog-client-go/auth"
"github.com/jfrog/jfrog-client-go/http/jfroghttpclient"
"github.com/jfrog/jfrog-client-go/utils"
"github.com/jfrog/jfrog-client-go/utils/errorutils"
"net/http"
)

const (
ConfigProfileMinXscVersion = "1.11.0"
xscConfigProfileApi = "api/v1/profile"
)

type ConfigurationProfileService struct {
client *jfroghttpclient.JfrogHttpClient
XscDetails auth.ServiceDetails
}

func NewConfigurationProfileService(client *jfroghttpclient.JfrogHttpClient) *ConfigurationProfileService {
return &ConfigurationProfileService{client: client}
}

type ConfigProfile struct {
ProfileName string `json:"profile_name"`
FrogbotConfig FrogbotConfig `json:"frogbot_config,omitempty"`
Modules []Module `json:"modules"`
IsDefault bool `json:"is_default,omitempty"`
}

type FrogbotConfig struct {
EmailAuthor string `json:"email_author,omitempty"`
AggregateFixes bool `json:"aggregate_fixes,omitempty"`
AvoidPreviousPrCommentsDeletion bool `json:"avoid_previous_pr_comments_deletion,omitempty"`
BranchNameTemplate string `json:"branch_name_template,omitempty"`
PrTitleTemplate string `json:"pr_title_template,omitempty"`
PrCommentTitle string `json:"pr_comment_title,omitempty"`
CommitMessageTemplate string `json:"commit_message_template,omitempty"`
ShowSecretsAsPrComment bool `json:"show_secrets_as_pr_comment,omitempty"`
}

type Module struct {
ModuleId int32 `json:"module_id,omitempty"`
ModuleName string `json:"module_name"`
PathFromRoot string `json:"path_from_root"`
ReleasesRepo string `json:"releases_repo,omitempty"`
AnalyzerManagerVersion string `json:"analyzer_manager_version,omitempty"`
AdditionalPathsForModule []string `json:"additional_paths_for_module,omitempty"`
ExcludePaths []string `json:"exclude_paths,omitempty"`
ScanConfig ScanConfig `json:"scan_config"`
ProtectedBranches []string `json:"protected_branches,omitempty"`
IncludeExcludeMode int32 `json:"include_exclude_mode,omitempty"`
IncludeExcludePattern string `json:"include_exclude_pattern,omitempty"`
ReportAnalytics bool `json:"report_analytics,omitempty"`
}

type ScanConfig struct {
ScanTimeout int32 `json:"scan_timeout,omitempty"`
ExcludePattern string `json:"exclude_pattern,omitempty"`
EnableScaScan bool `json:"enable_sca_scan,omitempty"`
EnableContextualAnalysisScan bool `json:"enable_contextual_analysis_scan,omitempty"`
SastScannerConfig SastScannerConfig `json:"sast_scanner_config,omitempty"`
SecretsScannerConfig SecretsScannerConfig `json:"secrets_scanner_config,omitempty"`
IacScannerConfig IacScannerConfig `json:"iac_scanner_config,omitempty"`
ApplicationsScannerConfig ApplicationsScannerConfig `json:"applications_scanner_config,omitempty"`
ServicesScannerConfig ServicesScannerConfig `json:"services_scanner_config,omitempty"`
}

type SastScannerConfig struct {
EnableSastScan bool `json:"enable_sast_scan,omitempty"`
Language string `json:"language,omitempty"`
ExcludePatterns []string `json:"exclude_patterns,omitempty"`
ExcludeRules []string `json:"exclude_rules,omitempty"`
}

type SecretsScannerConfig struct {
EnableSecretsScan bool `json:"enable_secrets_scan,omitempty"`
ExcludePatterns []string `json:"exclude_patterns,omitempty"`
}

type IacScannerConfig struct {
EnableIacScan bool `json:"enable_iac_scan,omitempty"`
ExcludePatterns []string `json:"exclude_patterns,omitempty"`
}

type ApplicationsScannerConfig struct {
EnableApplicationsScan bool `json:"enable_applications_scan,omitempty"`
ExcludePatterns []string `json:"exclude_patterns,omitempty"`
}

type ServicesScannerConfig struct {
EnableServicesScan bool `json:"enable_services_scan,omitempty"`
ExcludePatterns []string `json:"exclude_patterns,omitempty"`
}

func (cp *ConfigurationProfileService) GetConfigurationProfile(profileName string) (*ConfigProfile, error) {
httpDetails := cp.XscDetails.CreateHttpClientDetails()
url := fmt.Sprintf("%s%s/%s", utils.AddTrailingSlashIfNeeded(cp.XscDetails.GetUrl()), xscConfigProfileApi, profileName)
res, body, _, err := cp.client.SendGet(url, true, &httpDetails)
if err != nil {
return nil, fmt.Errorf("failed to send GET query to '%s': %q", url, err)
}
if err = errorutils.CheckResponseStatusWithBody(res, body, http.StatusOK); err != nil {
return nil, err
}

var profile ConfigProfile
err = errorutils.CheckError(json.Unmarshal(body, &profile))
return &profile, err
eranturgeman marked this conversation as resolved.
Show resolved Hide resolved
}
Loading