Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes #104 by supporting properties via environment variables #108

Merged
merged 3 commits into from
Aug 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
*.class
*.iml
*.jar
/.calva/output-window/
/.calva/repl.calva-repl
/.clj-kondo/.cache/
/.cpcache/
/.idea/
Expand Down
69 changes: 63 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ clojure -M:clj-watson scan -p deps.edn
The first time it runs, it will download the entire vulnerability database, which
can take several minutes. Subsequent runs will be much faster.

> [!NOTE]
> [!NOTE]
> The database is stored in the `/tmp/db/` folder (on macOS/Linux) - in case you ever need to delete that folder, if it looks like the database is corrupted.
`clj-watson` can also be installed as a Clojure CLI tool:
Expand Down Expand Up @@ -105,7 +105,8 @@ It is easy to [request an API key](https://github.com/jeremylong/DependencyCheck
You can specify you key via:

1. The `nvd.api.key` Java system property on the command line
2. Or, an `nvd.api.key` entry in your `clj-watson.properties` file
2. Or, the `CLJ_WATSON_NVD_API_KEY` environment variable
3. Or, an `nvd.api.key` entry in your `clj-watson.properties` file

> [!CAUTION]
> Keeping your nvd api key secret is your responsibility.
Expand All @@ -129,7 +130,39 @@ clojure -J-Dnvd.api.key=<your key here> -Tclj-watson scan :p deps.edn
Replace `<your key here>` with your actual api key.

> [!CAUTION]
> You could specify this system property under `:jvm-opts` in your `deps.edn` under your `:clj-watson` alias, but be careful not to commit it to version control.
> You could specify this system property under `:jvm-opts` in your `deps.edn` under your `:clj-watson` alias, but be careful not to commit it to version control.
##### Via Environment Variable

Example usage:

```shell
CLJ_WATSON_NVD_API_KEY=<your key here> clojure -M:clj-watson scan -p deps.edn
```

Or:

```shell
export CLJ_WATSON_NVD_API_KEY=<your key here>

clojure -M:clj-watson scan -p deps.edn
```

Or:

```shell
CLJ_WATSON_NVD_API_KEY=<your key here> clojure -Tclj-watson scan :p deps.edn
```

Or:

```shell
export CLJ_WATSON_NVD_API_KEY=<your key here>

clojure -Tclj-watson scan :p deps.edn
```

Replace `<your key here>` with your actual api key.

##### Via the `clj-watson.properties` File

Expand All @@ -154,7 +187,7 @@ Or:
clojure -Tclj-watson scan :p deps.edn :clj-watson-properties ./clj-watson.properties
```

> [!CAUTION]
> [!CAUTION]
> Be careful not to commit your key to version control.
### GitHub Advisory Database [experimental]
Expand Down Expand Up @@ -233,7 +266,7 @@ the `--suggest-fix` or `-s` option when running `clj-watson`.
# Installation

> [!IMPORTANT]
> You'll need to [setup your NVD API key](#nist-nvd-api).
> You'll need to [setup your NVD API key](#nist-nvd-api).
`clj-watson` can be installed as a Clojure CLI tool, as shown above. While
this is the easiest way to install the latest version and keep it up-to-date
Expand Down Expand Up @@ -309,6 +342,8 @@ OPTIONS valid when database-strategy is dependency-check:
See docs for configuration. [false]
```

## Properties

By default, when using the DEPENDENCY-CHECK strategy, `clj-watson` will load
its own `dependency-check.properties` file, and then look for a
`clj-watson.properties` file on the classpath and load that if found, for
Expand All @@ -324,14 +359,35 @@ that file and apply those properties to the dependency-check scan. This is
in addition to the properties loaded from the `dependency-check.properties`
or the `-d` file. This can be useful to override just a few properties.

In addition, relevant properties provided as Java system properties are
read by the underlying DependencyCheck scan, and take precedence over the
properties provided in these files. See the `-Dnvd.api.key=` example above.

## Environment Variables

`clj-watson` also supports environment variables that start with `CLJ_WATSON_`.
These are used to set properties that are not provided on the command line.
The `CLJ_WATSON_` prefix is removed, and the rest of the name is converted to
a lowercase property name with `_` replaced by `.` (e.g., `CLJ_WATSON_NVD_API_KEY`).
To specify a property name that contains an underscore, use two underscores
in the environment variable name, e.g., `CLJ_WATSON_DATA_FILE__NAME` to
set the `data.file_name` property.

Properties set via environment variables take precedence over those set in
the properties files described above, but not over Java system properties
set on the command-line.

Environment variables are often the most straightforward and most secure
way to provide sensitive information like API keys in various CI systems.

# Execution

The minimum needed to run `clj-watson` is to provide the path to a `deps.edn`
file, but it is recommended that you also provide the `-s` option so
`clj-watson` will try to suggest remediations for any vulnerabilities found.

> [!IMPORTANT]
> You'll need to first [setup your NVD API key](#nist-nvd-api).
> You'll need to first [setup your NVD API key](#nist-nvd-api).
```bash
clojure -M:clj-watson -p deps.edn
Expand Down Expand Up @@ -383,6 +439,7 @@ It writes settings and vulnerability findings to `stdout`.
# Who uses it

- [180 Seguros](https://180s.com.br)
- [org.clojure/tools.deps](https://github.com/clojure/tools.deps)
- [World Singles Networks](https://worldsinglesnetworks.com/)

# Development
Expand Down
43 changes: 34 additions & 9 deletions src/clj_watson/controller/dependency_check/scanner.clj
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,40 @@
;; ensure (fake) API key is redacted:
(sanitize-property "nvd.api.key=72a48765-90ab-5678-abcd-1234abcd5489"))

(defn ^:private env-var->property [env-var]
(-> env-var
(string/replace #"^CLJ_WATSON_" "") ; strip prefix
(string/lower-case) ; lowercase
(string/replace "_" ".") ; _ -> .
(string/replace ".." "_"))) ; allow __ -> .. -> _

(comment
(env-var->property "CLJ_WATSON_NVD_API_KEY")
(env-var->property "CLJ_WATSON_DATA_FILE__NAME"))

(defn ^:private set-watson-env-vars-as-properties []
(run! (fn [[env-var value]]
(when (string/starts-with? env-var "CLJ_WATSON_")
(let [property (env-var->property env-var)
p-value (System/getProperty property)]
(if p-value
(println (str "Ignoring " env-var " as " property " is already set."))
(do
(println (str "Setting " property " from " env-var "."))
(System/setProperty property value))))))
Comment on lines +40 to +44
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the feedback to the user here!

(System/getenv))
(println))

(defn ^:private create-settings [^String properties-file-path ^String additional-properties-file-path]
(let [settings (Settings.)]
(let [props
(if properties-file-path
(->> properties-file-path File.)
(->> "dependency-check.properties" io/resource))]
(println (str "\nRead " (count (line-seq (io/reader props))) " dependency-check properties."))
(if properties-file-path
(->> props (.mergeProperties settings))
(->> props slurp .getBytes ByteArrayInputStream. (.mergeProperties settings))))
(let [settings (Settings.)
props
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nested let was bothering me 🙂

(if properties-file-path
(->> properties-file-path File.)
(->> "dependency-check.properties" io/resource))]
(println (str "\nRead " (count (line-seq (io/reader props))) " dependency-check properties."))
(if properties-file-path
(->> props (.mergeProperties settings))
(->> props slurp .getBytes ByteArrayInputStream. (.mergeProperties settings)))
(if-let [add-props
(if additional-properties-file-path
(->> additional-properties-file-path File.)
Expand All @@ -50,6 +74,7 @@
(->> add-props (.mergeProperties settings))
(some->> add-props slurp .getBytes ByteArrayInputStream. (.mergeProperties settings))))
(println "No additional properties found.\n"))
(set-watson-env-vars-as-properties)
settings))

(defn ^:private validate-settings
Expand Down