Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: Adding documentation website #377

Merged
merged 15 commits into from
Nov 21, 2024
20 changes: 20 additions & 0 deletions documentation/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Dependencies
/node_modules

# Production
/build

# Generated files
.docusaurus
.cache-loader

# Misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local

npm-debug.log*
yarn-debug.log*
yarn-error.log*
41 changes: 41 additions & 0 deletions documentation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Website

This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.

### Installation

```
$ yarn
```

### Local Development

```
$ yarn start
```

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

### Build

```
$ yarn build
```

This command generates static content into the `build` directory and can be served using any static contents hosting service.

### Deployment

Using SSH:

```
$ USE_SSH=true yarn deploy
```

Not using SSH:

```
$ GIT_USER=<Your GitHub username> yarn deploy
```

If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
9 changes: 9 additions & 0 deletions documentation/docs/Develop/contributing.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
position: 1
title: Contributing
---

# Contriubting
**Thank you for wanting to contribute to OWASP dep-scan!**

There are a lot of ways to contribute to dep-scan, we are in constant need for developers to help with writing bug fixes, additional documentation or writing blogs for our website.
18 changes: 18 additions & 0 deletions documentation/docs/Develop/core-concepts.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
position: 4
title: Core Concepts and Modules
---

# Core Concepts and Modules

## Key Concepts

:::info
Examples will be included here
:::

## Module Documentation

:::info
Examples will be included here
:::
38 changes: 38 additions & 0 deletions documentation/docs/Develop/debugging.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
position: 3
title: Debugging
---

# Debugging

This page teachs users to setup python debugger for dep-scan project.

## Debugging using Pycharm

1. Open `dep-scan` using pycharm, when you open the project for the first time it will take some time initializing everything.
![1](/img/develop/debug/1.png)
2. Procede by updating your python environment set in Pycharm.
a. Click on Python3.x on the bottom right corner of Pycharm
b. Choose "Add New Interpreter" > "Add Local Interpreter".
c. Choose `Virtualenv Environment` and existing and find the `venv` folder inside of dep-scan.(Can be different depending on your system)
![1](/img/develop/debug/2.png)
3. You may add a debugger point to any part of dep-scan code. For example we are pointing it to `main()` function in `dep-scan/cli.py` as this is activated whenever we run dep-scan using the cli.
![1](/img/develop/debug/3.png)
4. Go to Pycharms top bar and click on "Current File" > "Edit Configurations...".
![1](/img/develop/debug/4.png)
5. Add a new configurations and choose "Python".
![1](/img/develop/debug/5.png)
6. In the following dialog:
a. add a name to your config
b. then next to script button choose the `cli.py` file.
c. add arguments that you want to test.
d. finally click on save.
![1](/img/develop/debug/6.png)
7. Press the debug button.
![1](/img/develop/debug/7.png)
8. Success!! 🥳
![1](/img/develop/debug/8.png)

{/* ## Debugging using VSCode

## Debugging using PDB */}
18 changes: 18 additions & 0 deletions documentation/docs/Develop/develop.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: Develop
---

# Development Guide

## Developer's Introduction to Dep-scan

### Project Goals and Scope

### Tech Stack

### Architecture Diagram


{/*import DocCardList from '@theme/DocCardList';

<DocCardList /> */}
53 changes: 53 additions & 0 deletions documentation/docs/Develop/getting-started-development.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
position: 2
title: Getting Started with Development
---

# Getting Started (Development)

OWASP dep-scan is under very active development by AppThreat and the OWASP community.
This is a comprehensive guide to contributing for developers of all experience level.

## Setting up the Development Environment
Here are steps to download and run dep-scan software.

:::note
dep-scan utilizes the `Cyclonedx/cdxgen` project. Therefore is it required to install it using
```bash
npm install -g @cyclonedx/cdxgen
```
You would have to download any package managers utilized by your project on your system in order to ensure that cdxgen works on them properly.

:::tip
While using cdxgen it is a good idea to `export CDXGEN_DEBUG_MODE=debug` to ensure if you can install and package managers you might be missing for a scan.
:::


1. Clone `owasp-dep-scan/dep-scan` project repository.

```bash
git clone https://github.com/owasp-dep-scan/dep-scan
cd dep-scan
```

2. Setup and activate a python environment of your choice. We use `venv` to manager our environemnts as an example here.

```bash
python3 -m venv venv
source venv/bin/activate
```

3. Install dependencies and building dep-scan.

```bash
pip install -e .
```

:::tip
`-e` flag in pip creates a local install which is means that changes made to the project can be used instantly without having to install the project again.
:::

{/* ## Code Style Guide

## Contribution Workflow */}

6 changes: 6 additions & 0 deletions documentation/docs/Develop/help.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
position: 6
title: Help
---

For more help regarding anything
13 changes: 13 additions & 0 deletions documentation/docs/Develop/testing-quality.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
position: 5
title: Testing and QA
---

# Testing and QA

## Testing Strategies

## Code Coverage Guidelines

## CI/CD

147 changes: 147 additions & 0 deletions documentation/docs/adv-usage.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
---
sidebar_position: 8
---

# Advanced Usage

## GitHub Security Advisory

To download security advisories from GitHub, a personal access token with minimal permissions is necessary.

- Fine-grained token: Grant no permissions and select the following for repository access: `Public Repositories (read-only)`
- Token (classic): Grant no permissions

```bash
export GITHUB_TOKEN="<PAT token>"
```

## Suggest mode

Depscan comes with a suggest mode enabled by default to simplify the triaging experience. The fix version for each vulnerability is retrieved from the sources. Sometimes, there might be known vulnerabilities in the fix version reported. Eg: in the below screenshot the fix versions suggested for jackson-databind might contain known vulnerabilities.

[//]: # (![Normal mode]&#40;docs/depscan-normal.png&#41;)

[//]: # (![Suggest mode]&#40;docs/depscan-suggest.png&#41;)

Notice, how the new suggested version is `2.9.10.5` which is an optimal fix version. Please note that the optimal fix version may not be the appropriate version for your application based on compatibility.

Pass `--no-suggest` to disable this behavior.

## Package Risk audit

`--risk-audit` argument enables package risk audit. Currently, only npm and PyPI packages are supported in this mode. Some risk factors are identified and assigned weights to compute a final risk score. Packages that then exceed a maximum risk score (`config.pkg_max_risk_score`) are presented in a table.

Use `--private-ns` to specify the private package namespace that should be checked for dependency confusion type issues where a private package is available on the public npm/pypi registry.

For example, to check if private packages with namespaces @appthreat and @shiftleft are not accidentally made public, use the below argument.

```
--private-ns appthreat,shiftleft
```

| Risk category | Default Weight | Reason |
| ------------------------------ | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| pkg_private_on_public_registry | 4 | Private package is available on a public registry |
| pkg_min_versions | 2 | Packages with less than 3 versions represent an extreme where they could be either super stable or quite recent. Special heuristics are applied to ignore older stable packages |
| mod_create_min_seconds | 1 | Less than 12 hours difference between modified and creation time. This indicates that the upload had a defect that had to be rectified immediately. Sometimes, such a rapid update could also be malicious |
| latest_now_min_seconds | 0.5 | Less than 12 hours difference between the latest version and the current time. Depending on the package such a latest version may or may not be desirable |
| latest_now_max_seconds | 0.5 | Package versions that are over 6 years old are in use. Such packages might have vulnerable dependencies that are known or yet to be found |
| pkg_min_maintainers | 2 | Package has less than 2 maintainers. Many opensource projects have only 1 or 2 maintainers so special heuristics are used to ignore older stable packages |
| pkg_min_users | 0.25 | Package has less than 2 npm users |
| pkg_install_scripts | 2 | Package runs a custom pre or post installation scripts. This is often malicious and a downside of npm. |
| pkg_node_version | 0.5 | Package supports outdated version of node such as 0.8, 0.10, 4 or 6.x. Such projects might have prototype pollution or closure related vulnerabilities |
| pkg_scope | 4 or 0.5 | Packages that are used directly in the application (required scope) gets a score with a weight of 4. Optional packages get a score of 0.25 |
| deprecated | 1 | Latest version is deprecated |

Refer to `pkg_query.py::get_category_score` method for the risk formula.

### Automatic adjustment

A parameter called `created_now_quarantine_seconds` is used to identify packages that are safely past the quarantine period (1 year). Certain risks such as `pkg_min_versions` and `pkg_min_maintainers` are suppressed for packages past the quarantine period. This adjustment helps reduce noise since it is unlikely that a malicious package can exist in a registry unnoticed for over a year.

### Configuring weights

All parameters can be customized by using environment variables. For eg:

export&nbsp;`PKG_MIN_VERSIONS=4` to increase and set the minimum versions category to 4.

## Live OS scan

By passing `-t os`, depscan can generate an SBOM for a live operating system or a VM with OS packages and kernel information. Optionally, pass the argument `--deep` to generate an SBOM with both OS and application packages and to check for application vulnerabilities.

All OS packages.

```bash
depscan -t os -i . -o reports/depscan.json
```

All OS and application packages.

```bash
depscan -t os --deep -i . -o reports/depscan.json
```

## License scan

dep-scan can scan the dependencies for any license limitations and report them directly on the console log. To enable license scanning set the environment variable `FETCH_LICENSE` to `true`.

```
export FETCH_LICENSE=true
```

The license data is sourced from choosealicense.com and is quite limited. If the license of a given package cannot be reliably matched against this list it will get silently ignored to reduce any noise. This behavior could change in the future once the detection logic gets improved.

[//]: # (![License scan]&#40;docs/license-scan.png&#41;)

## Kubernetes and Cloud apps

dep-scan could auto-detect most cloud applications and Kubernetes manifest files. Pass the argument `-t yaml-manifest` to manually specify the type.

## PDF reports

Ensure [wkhtmltopdf](https://wkhtmltopdf.org/downloads.html) is installed or use the official container image to generate pdf reports. Use with `--explain` for more detailed reports.

## Custom reports

dep-scan can be provided with a [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) template using the `--report-template` parameter.
Giving it will pass the vulnerability report into your template for rendering the report.

Please find a basic example here:

```jinja
{% if metadata -%}
Report for {{ metadata.component.group }}:{{ metadata.component.name }}:{{ metadata.component.version }}
{% endif -%}

{% if vulnerabilities -%}
There were {{ vulnerabilities | length }} issues identified:

{% for vuln in vulnerabilities -%}
* {{ vuln['bom-ref'] }} - {{ vuln.recommendation }}
{% endfor -%}
{% else -%}
🏆 _No vulnerabilities found_
{% endif -%}

Severity counts:
* Low: {{ summary.LOW }}
* Medium: {{ summary.MEDIUM }}
* High: {{ summary.HIGH }}
* Critical: {{ summary.CRITICAL }}
* Unspecified: {{ summary.UNSPECIFIED }}
```

The objects available are taken from the CycloneDX \*.vdr.json BOM file generated, just have a look at the file for its full structure:

- `metadata`
- `vulnerabilities`
- `components`
- `dependencies`
- `services`

`summary` is a dictionary type with vulnerability severity quantities as shown in the example above.
`pkg_vulnerabilities` - Same as `vulnerabilities` from the VDR
`pkg_group_rows` - List of vulnerability id and the dependency tree prioritized by depscan.

Furthermore, insights are imaginable to be made available to the template, please reach out or contribute on demand.
We appreciate it if you like to contribute your report templates as examples, please add/find them [here](https://github.com/owasp-dep-scan/dep-scan/blob/master/contrib/report-templates).
Loading