Skip to content

Commit

Permalink
feat: Removes mutation of .pre-commit-config.yaml by seCureLI (#264)
Browse files Browse the repository at this point in the history 
This PR mostly deletes code related to mutating .pre-commit-config.yaml
based on entries in .secureli.yaml.

Once merged, users will be able to update .pre-commit-config.yaml
directly to modify the behavior of the pre-commit hooks.
  • Loading branch information
@stujfiter
stujfiter authored Aug 9, 2023
1 parent 8005f00 commit fc3c6ff
Show file tree
Hide file tree
Showing 15 changed files with 98 additions and 1,659 deletions.
47 changes: 1 addition & 46 deletions .pre-commit-config.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,46 +1 @@
repos:
- hooks:
- id: python-use-type-annotations
repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.9.0
- hooks:
- args:
- --exclude
- tests/
- --severity-level
- medium
id: bandit
repo: https://github.com/PyCQA/bandit
rev: 1.7.4
- hooks:
- id: black
repo: https://github.com/psf/black
rev: 22.10.0
- hooks:
- id: check-added-large-files
- id: check-ast
- id: check-docstring-first
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: check-merge-conflict
- id: check-toml
- id: check-json
- id: check-xml
- id: check-yaml
- id: debug-statements
- args:
- --allow-missing-credentials
id: detect-aws-credentials
- id: detect-private-key
- args:
- --pytest-test-first
id: name-tests-test
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
- hooks:
- id: detect-secrets
repo: https://github.com/Yelp/detect-secrets
rev: v1.4.0
repos: []
48 changes: 12 additions & 36 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,13 @@ pip install secureli

Once installed you can see the latest documentation for seCureLI by entering the following on a command prompt:

```python
```bash
% secureli --help
```

You will see a list of commands and descriptions of each. You can also pull up documentation for each command with the same pattern. For example:

```python
```bash
% secureli init --help

Usage: secureli init [OPTIONS]
Expand All @@ -62,7 +62,7 @@ You will see a list of commands and descriptions of each. You can also pull up d
When invoking these commands, you can combine the short versions into a single flag. For example, the following commands are equivalent:
```python
```bash
% secureli init --reset --yes
% secureli init -ry
```
Expand All @@ -72,7 +72,7 @@ After seCureLI is installed, you can use it to configure your local git reposito
All you need to do is run:
```commandline
```bash
% secureli init
```
Expand Down Expand Up @@ -110,50 +110,26 @@ seCureLI is configurable via a .secureli.yaml file present in the root of your l
### top level
| Key | Description |
| ------------------ |----------------------------------------------------------------------------------------------------------------------------------|
| `repo_files` | Affects how seCureLI will interpret the repository, both for language analysis and as it executes various linters. |
| `echo` | Adjusts how seCureLI will print information to the user. |
| `language_support` | Affects seCureLI's language analysis and support phase. |
| `pre_commit` | Enables various overrides and options for seCureLI's configuration and usage of pre-commit, the underlying code analysis system. |
| Key | Description |
|--------------------|--------------------------------------------------------------------------------------------------------------------|
| `repo_files` | Affects how seCureLI will interpret the repository, both for language analysis and as it executes various linters. |
| `echo` | Adjusts how seCureLI will print information to the user. |
| `language_support` | Affects seCureLI's language analysis and support phase. |
### repo_files
| Key | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `max_file_size` | A number in bytes. Files over this size will not be considered during language analysis, for speed purposes. Default: 100000 |
| `ignored_file_extensions` | Which file extensions not to consider during language analysis. |
| `exclude_file_patterns` | Which file patterns to ignore during language analysis and code analysis execution. Use a typical file pattern you might find in a .gitignore file, such as `*.py` or `tests/`. Certain patterns you will have to wrap in double-quotes for the entry to be valid YAML. |
### echo
| Key | Description |
| ------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|---------|----------------------------------------------------------------------------------------------------------------------------------------------------|
| `level` | The log level to display to the user. Defaults to ERROR, which includes `error` and `print` messages, without including warnings or info messages. |
### pre_commit

| Key | Description |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `repos` | A set of template-based Pre-Commit Repos to configure with overrides, identified by URL. These override repo-configurations stored in the template, and attempting to modify a repo not configured into the template will have no effect. |
| `suppressed_repos` | A set of template-based Pre-Commit Repo URLs to completely remove from the final configuration. These remove repo configurations stored in the template, removing a repo not stored in the template will be ignored. |

### pre_commit.repos

| Key | Description |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `url` | The identifying URL of the repo being leveraged by pre-commit, within which one or more hooks can be leveraged. |
| `hooks` | A set of hooks associated with the specified repository to override. See the next section for what we can configure there. |
| `suppressed_hook_ids` | A set of hook IDs to remove from the repository as configured within the template. Hook IDs not present in the template configuration will be ignored. |

### pre_commit.repos.hooks

| Key | Description |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | The identifying string of the pre-commit hook to override. |
| `arguments` | A set of arguments to provide to the pre-commit hook identified by `id`. These arguments overwrite any existing arguments. |
| `additional_args` | A set of arguments to provide to the pre-commit hook identified by `id`. These arguments are appended after an existing arguments. |
| `exclude_file_patterns` | A set of file patterns to provide to pre-commit to ignore for the purposes of this hook. Use a typical file pattern you might find in a .gitignore file, such as `*.py` or `tests/`. Certain patterns you will have to wrap in double-quotes for the entry to be valid YAML. |
## Using Observability Platform to Show Secret Detection Statistics
Expand All @@ -168,7 +144,7 @@ Should you need seCureLI to work with other platforms, please create a new issue
- Once the above setup is complete, everytime seCureLI triggered, it should send a usage log to New Relic
- In New Relic, you can create a dashboard of metric to see the number of times secret was caught using query such as
```commandline
```pre
FROM Log Select sum(failure_count_details.detect_secrets) as 'Caught Secret Count'
```
Expand Down
67 changes: 1 addition & 66 deletions secureli/actions/action.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
from secureli.abstractions.echo import EchoAbstraction, Color
from secureli.abstractions.pre_commit import (
InstallFailedError,
PreCommitAbstraction,
)
from secureli.repositories.secureli_config import (
SecureliConfig,
Expand All @@ -17,10 +16,10 @@
)
from secureli.repositories.settings import SecureliRepository
from secureli.services.language_analyzer import LanguageAnalyzerService, AnalyzeResult
from secureli.services.language_config import LanguageNotSupportedError
from secureli.services.language_support import LanguageSupportService
from secureli.services.scanner import ScannerService, ScanMode
from secureli.services.updater import UpdaterService
from secureli.services.language_config import LanguageNotSupportedError


class VerifyOutcome(str, Enum):
Expand Down Expand Up @@ -102,26 +101,6 @@ def verify_install(
if not config.languages or not config.version_installed:
return self._install_secureli(folder_path, always_yes)
else:
available_version = self.action_deps.language_support.version_for_language(
config.languages
)

# Check for a new version and prompt for upgrade if available
if available_version != config.version_installed:
return self._upgrade_secureli(config, available_version, always_yes)

# Validates the current .pre-commit-config.yaml against the generated config
config_validation_result = (
self.action_deps.language_support.validate_config(
languages=config.languages
)
)

# If config mismatch between available version and current version prompt for upgrade
if not config_validation_result.successful:
self.action_deps.echo.print(config_validation_result.output)
return self._update_secureli(always_yes)

self.action_deps.echo.print(
f"seCureLI is installed and up-to-date (languages = {config.languages})"
)
Expand All @@ -130,50 +109,6 @@ def verify_install(
config=config,
)

def _upgrade_secureli(
self, config: SecureliConfig, available_version: str, always_yes: bool
) -> VerifyResult:
"""
Installs seCureLI into the given folder path and returns the new configuration
:param config: The existing configuration for seCureLI
:param available_version: The new version we're upgrading to
:param always_yes: Assume "Yes" to all prompts
:return: The new SecureliConfig after upgrade or None if upgrading did not complete
"""
self.action_deps.echo.print(
f"The config version installed is {config.version_installed}, but the latest is {available_version}"
)
response = always_yes or self.action_deps.echo.confirm(
"Upgrade now?",
default_response=True,
)
if not response:
self.action_deps.echo.warning("User canceled upgrade process")
return VerifyResult(
outcome=VerifyOutcome.UPGRADE_CANCELED,
config=config,
)

try:
metadata = self.action_deps.language_support.apply_support(config.languages)

# Update config with new version installed and save it
config.version_installed = metadata.version
self.action_deps.secureli_config.save(config)
self.action_deps.echo.print("seCureLI has been upgraded successfully")
return VerifyResult(
outcome=VerifyOutcome.UPGRADE_SUCCEEDED,
config=config,
)
except InstallFailedError:
self.action_deps.echo.error(
"seCureLI could not be upgraded due to an error"
)
return VerifyResult(
outcome=VerifyOutcome.UPGRADE_FAILED,
config=config,
)

def _install_secureli(self, folder_path: Path, always_yes: bool) -> VerifyResult:
"""
Installs seCureLI into the given folder path and returns the new configuration
Expand Down
Loading

0 comments on commit fc3c6ff

Please sign in to comment.