Skip to content

Commit

Permalink
Merge pull request #12 from theztefan/main
Browse files Browse the repository at this point in the history
Adding Secret Scanning out-of-the-box validity field in output
  • Loading branch information
CallMeGreg committed Aug 19, 2024
2 parents 09101a1 + f56c499 commit 3d03d85
Show file tree
Hide file tree
Showing 7 changed files with 77 additions and 41 deletions.
Binary file removed .DS_Store
Binary file not shown.
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
/gh-secret-scanning
/gh-secret-scanning.exe
release-command.md
Secret Scanning Report*.csv
secret*.csv
Secret*.csv
.DS_Store
62 changes: 38 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,88 +1,102 @@
# Overview
# gh-secret-scanning - The GitHub Secret Scanning CLI Extension

This project is a GitHub CLI (`gh`) extension that provides commands for interacting with secret scanning alerts.

This extension helps GitHub Advanced Security (GHAS) customers prioritize remediation of their secret scanning alerts by identifying and focusing on those that are confirmed active first.

While this extension works for Enterprise Cloud (GHEC) customers, it is primarily intended for GitHub Enterprise Server (GHES) customers who do not have access to the [GitHub.com secret scanning validity check feature](https://docs.github.com/en/enterprise-cloud@latest/code-security/secret-scanning/managing-alerts-from-secret-scanning#validating-partner-patterns).
While this extension works for Enterprise Cloud (GHEC) customers, it is primarily intended for GitHub Enterprise Server (GHES) customers who do not have access to the [GitHub.com secret scanning validity check feature](https://docs.github.com/en/enterprise-cloud@latest/code-security/secret-scanning/managing-alerts-from-secret-scanning/evaluating-alerts#checking-a-secrets-validity). Validity check on GHES is available as of `>=3.12` but currently limited to GitHub Personal Access Tokens (PAT).

Primary features include:

- Listing secret scanning alerts for an enterprise, organization, or repository
- Verifying if secret scanning alerts are still active
- Expand the out-of-the-box secret scanning validity checks capabilities with custom validators
- Opening issues in repos that contain valid secrets

# Supported Token Types
## Supported Token Types

- GitHub Personal Access Tokens (GHES + GHEC)
- Slack API Tokens

# Pre-requisites
## Pre-requisites

- [GitHub CLI](https://github.com/cli/cli#installation)
- [GHES 3.7+](https://docs.github.com/en/[email protected]/admin/all-releases#releases-of-github-enterprise-server) or [GHEC](https://docs.github.com/en/enterprise-cloud@latest/admin/overview/about-github-enterprise-cloud)
- [GHAS](https://docs.github.com/en/enterprise-cloud@latest/get-started/learning-about-github/about-github-advanced-security)

# Installation
```
## Installation

```bash
gh extension install CallMeGreg/gh-secret-scanning
```

# Usage
## Usage

Authenticate with your GitHub Enterprise Server or GitHub Enterprise Cloud account:
```

```bash
gh auth login
```

## Alerts subcommand
### Alerts subcommand

Target either an enterprise, organization, or repository by specifying the `-e`, `-o`, or `-r` flags respectively. _Exactly one selection from these three flags is required._

```
```bash
gh secret-scanning alerts -e <enterprise>
```

```
```bash
gh secret-scanning alerts -o <organization>
```

```
```bash
gh secret-scanning alerts -r <repository>
```

Optionally add flags to specify a GHES server, limit the number of secrets processed, filter for a specific secret provider, display the secret values, generate a csv report, include extra fields, and more:
```

```bash
gh secret-scanning alerts -e github --url my-github-server.com --limit 10 --provider slack --show-secret --csv --verbose
```

## Verify subcommand
### Verify subcommand

Target either an enterprise, organization, or repository by specifying the `-e`, `-o`, or `-r` flags respectively. _Exactly one selection from these three flags is required._

```
```bash
gh secret-scanning verify -e <enterprise>
```

```
```bash
gh secret-scanning verify -o <organization>
```

```
```bash
gh secret-scanning verify -r <repository>
```

Optionally add flags to specify a GHES server, limit the number of secrets processed, filter for a specific secret provider, display the secret values, generate a csv report, include extra fields, and more:
```

```bash
gh secret-scanning verify -e github --url my-github-server.com --limit 10 --provider slack --show-secret --csv --verbose
```

Also, optionally create an issue in any repository that contains a valid secret by using the `--create-issues` (`-i`) flag:
```

```bash
gh secret-scanning verify -e github --url my-github-server.com --create-issues
```

### Help

## Help
See available commands and flags by running:
```

```bash
gh secret-scanning -h
```

```
```bash
Interact with secret scanning alerts for a GHEC or GHES 3.7+ enterprise, organization, or repository

Usage:
Expand All @@ -109,8 +123,8 @@ Flags:
Use "secret-scanning [command] --help" for more information about a command.
```

# Demo
## Demo

This example first lists the alerts for an organization with the `alerts` subcommand, and then verifies the secrets with the `verify` subcommand. The `--csv` flag is used to generate a csv report of the results, and the `--create-issues` flag is used to create issues in any repository that contains a valid secret.

https://github.com/CallMeGreg/gh-secret-scanning/assets/110078080/fa8d7b08-1a2c-4522-ae96-5c3aab60107d

37 changes: 24 additions & 13 deletions cmd/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ type Alert struct {
Push_protection_bypassed bool `json:"push_protection_bypassed"`
Push_protection_bypassed_at string `json:"push_protection_bypassed_at"`
Push_protection_bypassed_by User `json:"push_protection_bypassed_by"`
Validity_github string `json:"validity"`
Validity_boolean bool `json:"validity_boolean"`
Validity_response_code string `json:"validity_response_code"`
Validity_endpoint string `json:"validity_endpoint"`
Expand Down Expand Up @@ -228,22 +229,14 @@ func addRepoFullNameToAlerts(alerts []Alert) []Alert {

func getSecretTypeParameter() (secret_type_param string) {
// if provider was specified, only return the secret types for that provider:
secret_type_param = "all"
if provider != "" {
// get keys for SupportedProviders[provider] map:
var secret_types []string
for key := range SupportedProviders[provider] {
secret_types = append(secret_types, key)
}
secret_type_param = strings.Join(secret_types, ",")
} else {
// get keys for every SupportedProvider key:
var secret_types []string
for _, supported_provider := range SupportedProviders {
for key := range supported_provider {
secret_types = append(secret_types, key)
}
}
secret_type_param = strings.Join(secret_types, ",")
}
return secret_type_param
}
Expand Down Expand Up @@ -272,6 +265,7 @@ func prettyPrintAlerts(alerts []Alert, validity_check bool) (err error) {
t.AddField("ID", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
t.AddField("State", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
t.AddField("Secret Type", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
t.AddField("Validity GitHub", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
if secret {
t.AddField("Secret", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
}
Expand Down Expand Up @@ -304,6 +298,7 @@ func prettyPrintAlerts(alerts []Alert, validity_check bool) (err error) {
t.AddField(strconv.Itoa(alert.Number), tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
t.AddField(alert.State, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
t.AddField(alert.Secret_type, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
t.AddField(alert.Validity_github, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
if secret {
t.AddField(alert.Secret, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
}
Expand Down Expand Up @@ -344,8 +339,9 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
counter := 0
// get current date & time:
now := time.Now()
timestamp := now.Format("2006-01-02 15-04-05")
filename := "Secret Scanning Report - " + scope + " - " + timestamp + ".csv"
// Format the time as YYYYMMDD-HHMMSS
timestamp := now.Format("20060102-150405")
filename := "secretscanningreport-" + scope + "-" + timestamp + ".csv"
// Create a CSV file
file, err := os.Create(filename)
if err != nil {
Expand All @@ -357,7 +353,7 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
writer := csv.NewWriter(file)
defer writer.Flush()
// Write headers to CSV file
headers := []string{"Repository", "ID", "State", "Secret Type"}
headers := []string{"Repository", "ID", "State", "Secret Type", "GitHub Validity"}
if secret {
headers = append(headers, "Secret")
}
Expand All @@ -371,7 +367,7 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
// Write data to CSV file
for counter < len(alerts) && counter < limit {
alert := alerts[counter]
row := []string{alert.Repository.Full_name, strconv.Itoa(alert.Number), alert.State, alert.Secret_type}
row := []string{alert.Repository.Full_name, strconv.Itoa(alert.Number), alert.State, alert.Secret_type, alert.Validity_github}
if secret {
row = append(row, alert.Secret)
}
Expand All @@ -393,9 +389,24 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
}

func verifyAlerts(alerts []Alert) (alertsOutput []Alert, err error) {
// Print Supported providers for reference when verbose flag is enabled
if verbose {
fmt.Println(Blue("Supported Providers:"))
for provider, secretTypes := range SupportedProviders {
for secretType := range secretTypes {
fmt.Printf("- %s - %s\n", provider, secretType)
}
}
}
for i, alert := range alerts {
// verify that the alert is valid by making a request to its validation endpoint:
provider := strings.Split(alert.Secret_type, "_")[0]

// Skip alert if provider is not supported
if _, ok := SupportedProviders[provider]; !ok {
continue
}

secret_type := alert.Secret_type
secret_validation_method := SupportedProviders[provider][secret_type]["HttpMethod"]
secret_validation_content_type := SupportedProviders[provider][secret_type]["ContentType"]
Expand Down
6 changes: 4 additions & 2 deletions cmd/verify.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,10 @@ func runVerify(cmd *cobra.Command, args []string) (err error) {
values.Set("per_page", per_page)
// if provider was specified, filter results for just that provider. Otherwise, target all supported providers:
secret_type := getSecretTypeParameter()
values.Set("secret_type", secret_type)
parsedURL.RawQuery = values.Encode()
if secret_type != "all" {
values.Set("secret_type", secret_type)
parsedURL.RawQuery = values.Encode()
}

// update the request path
requestPath = parsedURL.String()
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/CallMeGreg/gh-secret-scanning

go 1.19
go 1.23

require (
github.com/cli/go-gh v1.2.1
Expand Down
7 changes: 7 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,15 @@ github.com/cli/shurcooL-graphql v0.0.4/go.mod h1:3waN4u02FiZivIV+p1y4d0Jo1jc6BVi
github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542 h1:2VTzZjLZBgl62/EtslCrtky5vbi9dd7HrQPQIx6wqiw=
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
github.com/henvic/httpretty v0.0.6 h1:JdzGzKZBajBfnvlMALXXMVQWxWMF/ofTy8C3/OSUTxs=
github.com/henvic/httpretty v0.0.6/go.mod h1:X38wLjWXHkXT7r2+uK8LjCMne9rsuNaBLJ+5cU2/Pmo=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=
Expand All @@ -32,6 +35,7 @@ github.com/muesli/reflow v0.3.0/go.mod h1:pbwTDkVPibjO2kyvBQRBxTWEEGDGq0FlB1BIKt
github.com/muesli/termenv v0.13.0 h1:wK20DRpJdDX8b7Ek2QfhvqhRQFZ237RGRO0RQ/Iqdy0=
github.com/muesli/termenv v0.13.0/go.mod h1:sP1+uffeLaEYpyOTb8pLCUctGcGLnoFjSn4YJK5e2bc=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
Expand All @@ -42,6 +46,7 @@ github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyh
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/thlib/go-timezone-local v0.0.0-20210907160436-ef149e42d28e h1:BuzhfgfWQbX0dWzYzT1zsORLnHRv3bcRcsaUk0VmXA8=
github.com/thlib/go-timezone-local v0.0.0-20210907160436-ef149e42d28e/go.mod h1:/Tnicc6m/lsJE0irFMA0LfIwTBo4QP7A8IfyIv4zZKI=
golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
Expand All @@ -55,6 +60,8 @@ golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/h2non/gock.v1 v1.1.2 h1:jBbHXgGBK/AoPVfJh5x4r/WxIrElvbLel8TCZkkZJoY=
gopkg.in/h2non/gock.v1 v1.1.2/go.mod h1:n7UGz/ckNChHiK05rDoiC4MYSunEC/lyaUm2WWaDva0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

0 comments on commit 3d03d85

Please sign in to comment.