Bypass any plugin with square brackets

[tsigouris007]

• I'm submitting a ...
  • bug report

Current behavior

A user is able to bypass the tool engine by using [] characters. This is also a security issue.

Steps to reproduce

To reproduce you can use the following config.rb file as is (the secrets are dummy / generated values):

     1	client_1:
     2	  region: "eu-central-1"
     3	  access_key_id: "<%= ENV['CLIENT_ACCESS_KEY_ID'].presence || 'AKIA123456789ABCDEF1' %>"
     4	  secret_key: "<%= ENV['CLIENT_SECRET_KEY'].presence || 'izxN/tDSMytE0ZsmcG6JOts27fd1cIkkmzimFiL1' %>"
     5	client_2:
     6	  region: "eu-central-1"
     7	  access_key_id: "AKIA123456789ABCDEF2"
     8	  secret_key: "izxN/tDSMytE0ZsmcG6JOts27fd1cIkkmzimFiL2"
     9	client_3:
    10	  region: "eu-central-1"
    11	  access_key_id: "<%= ENV'CLIENT_ACCESS_KEY_ID'].presence || 'AKIA123456789ABCDEF3' %>"
    12	  secret_key: "<%= ENV'CLIENT_SECRET_KEY'].presence || 'izxN/tDSMytE0ZsmcG6JOts27fd1cIkkmzimFiL3' %>"

Or raw without the line numbers:

client_1:
  region: "eu-central-1"
  access_key_id: "<%= ENV['CLIENT_ACCESS_KEY_ID'].presence || 'AKIA123456789ABCDEF1' %>"
  secret_key: "<%= ENV['CLIENT_SECRET_KEY'].presence || 'izxN/tDSMytE0ZsmcG6JOts27fd1cIkkmzimFiL1' %>"
client_2:
  region: "eu-central-1"
  access_key_id: "AKIA123456789ABCDEF2"
  secret_key: "izxN/tDSMytE0ZsmcG6JOts27fd1cIkkmzimFiL2"
client_3:
  region: "eu-central-1"
  access_key_id: "<%= ENV'CLIENT_ACCESS_KEY_ID'].presence || 'AKIA123456789ABCDEF3' %>"
  secret_key: "<%= ENV'CLIENT_SECRET_KEY'].presence || 'izxN/tDSMytE0ZsmcG6JOts27fd1cIkkmzimFiL3' %>"

Run:

~$ detect-secrets-hook --json config.rb

Output:

  "results": {
    "config.rb": [
      {
        "type": "AWS Access Key",
        "filename": "config.rb",
        "hashed_secret": "778e2c0bca30ff1bf86f191e9117dd27c41ef7ad",
        "is_verified": false,
        "line_number": 7
      },
      {
        "type": "Base64 High Entropy String",
        "filename": "config.rb",
        "hashed_secret": "c89defacd77dcf2544bb89f70af22fa49b863c80",
        "is_verified": false,
        "line_number": 8
      },
      {
        "type": "Secret Keyword",
        "filename": "config.rb",
        "hashed_secret": "c89defacd77dcf2544bb89f70af22fa49b863c80",
        "is_verified": false,
        "line_number": 8
      },
      {
        "type": "AWS Access Key",
        "filename": "config.rb",
        "hashed_secret": "dee87956ca663e52348048bbafd68d9d94a58cce",
        "is_verified": false,
        "line_number": 11
      },
      {
        "type": "Base64 High Entropy String",
        "filename": "config.rb",
        "hashed_secret": "c982b61eb96a9c80e28c0da09f3ddcca122ab223",
        "is_verified": false,
        "line_number": 12
      }
    ]
  }

• Expected behavior

All access_key_ids and secret_keys should be caught.
The tool missed lines 3, 4.
By using the [] characters all secrets seem to be missed. Tried on multiple plugins.

• What is the motivation / use case for changing the behavior?

Security bypass.

• Environment:

  • detect-secrets Version: 1.5.0
  • Python Version: 3.9.2
  • OS Version: Debian 13
  • File type (if applicable): Any

• Other information

I suspected file detect_secrets/plugins/base.py method build_assignment_regex variables opt_open_square_bracket and opt_close_square_bracket and tried to solve locally with no success.
I have a feeling that the Generator is problematic.
Still analyzing the code and haven't pinpointed the exact location of the problem.

[tsigouris007]

So I found a workaround here.
In file detect_secrets/core/scan.py method _process_line_based_plugins I escaped the square brackets as shown:

line = line.rstrip().replace("[", "\[").replace("]", "\]")

I will open a PR if this works for you.
It didn't seem to break anything else but I haven't tested it thoroughly.

[tsigouris007]

Any improvements are more than welcome.
That was something that worked after testing.