Frogbot v2.24.1 changes behaviour for repositories with multiple Python packages

[williamvigolo-bmll]

Describe the bug

Hi, after the release of v2.24.1, we're seeing a breaking change in behaviour when a single repository contains multiple nested packages.

The repositories affected have one top-level package, and multiple nested packages. Something like this:

repository/
├── requirements.txt
├── setup.py
├── submodule1
│   ├── requirements.txt
│   └── setup.py
└── submodule2
    ├── requirements.txt
    └── setup.py

Current behavior

Previously Frogbot seems to only find and scan the top-level project (logs cut down for brevity):

Run jfrog/frogbot@v2
  with:
    version: latest
    ...
Frogbot
  /home/runner/_work/_tool/frogbot/[RELEASE]/x64/frogbot scan-repository
  14:59:26 [Info] Frogbot version: 2.24.0
  14:59:26 [Info] Running Frogbot "scan-repository" command
  14:59:31 [Info] Performing scans on 1 targets:
  [
    {
      "target": "/tmp/jfrog.cli.temp.-1737471567-3618821560",
      "technology": "pip"
    }
  ]

But with 2.24.1, Frogbot now discovers the subprojects automatically and tries to scan them:

Run jfrog/frogbot@v2
  with:
    version: latest
    ...
::group::Frogbot
Frogbot
  ##[debug]Obtaining an access token through OpenID Connect...
  ##[debug]Fetching JSON web token
  ##[debug]ID token url is ...
  ::add-mask::***
  ##[debug]Exchanging GitHub JSON web token with a JFrog access token...
  ::add-mask::***
  ##[debug]Downloading Frogbot from https://releases.jfrog.io/artifactory/frogbot/v2/[RELEASE]/frogbot-linux-amd64/frogbot
  ##[debug]Downloading https://releases.jfrog.io/artifactory/frogbot/v2/[RELEASE]/frogbot-linux-amd64/frogbot
  ##[debug]Destination /home/runner/_work/_temp/218448a2-ac41-4100-a458-b1882a52f75f
  ##[debug]download complete
  ##[debug]Caching tool frogbot [RELEASE] x64
  ##[debug]source file: /home/runner/_work/_temp/218448a2-ac41-4100-a458-b1882a52f75f
  ##[debug]destination /home/runner/_work/_tool/frogbot/[RELEASE]/x64
  ##[debug]destination file /home/runner/_work/_tool/frogbot/[RELEASE]/x64/frogbot
  ##[debug]finished caching tool
  /home/runner/_work/_tool/frogbot/[RELEASE]/x64/frogbot scan-pull-request
  15:53:02 [Info] Frogbot version: 2.24.1
  15:53:02 [Info] Running Frogbot "scan-pull-request" command
  15:53:03 [Info] Scanning Pull Request #5899 (from source branch: <.../...> to target branch: <.../master>)
  15:53:03 [Info] -----------------------------------------------------------
  15:53:03 [Info] ... repository downloaded successfully. Starting with repository extraction...
  15:53:06 [Info] Extracted repository successfully
  15:53:06 [Info] Scanning source branch...
  15:53:06 [Info] Performing scans on 3 targets:
  [
    {
      "target": "/tmp/jfrog.cli.temp.-1737474783-193021820/submodule1",
      "technology": "pip"
    },
    {
      "target": "/tmp/jfrog.cli.temp.-1737474783-193021820/submodule1",
      "technology": "pip"
    },
    {
      "target": "/tmp/jfrog.cli.temp.-1737474783-193021820",
      "technology": "pip"
    },
  ]

This is a breaking change, because Frogbot's behaviour has changed without a major version increase. In our case, the scanner failed to build the submodules because they require more than the standard pip build to build successfully.

We get an error like this:

Error: 8 [Error] target '/tmp/jfrog.cli.temp.-1737474783-193021820/submodule1 [pip]' errors:
  failed to build dependency tree: failed while building 'pip' dependency tree: "python -m pip install . -i ..." command failed: exit status 1 - Looking in indexes: ...

Followed by the usual pip build failure output.

Workaround: pin to 2.24.0 with:

      - uses: jfrog/[email protected]
        with:
          version: "2.24.0"

Reproduction steps

Create a nested Python project repository as described above. Run Frogbot v2.24.1, then v2.24.0 on the same repository. Notice that the former discovers the subpackages, while the latter only discovers the top-level package.

Expected behavior

v2.24.1 should not include this breaking change in behaviour compared to v2.24.0.

This new behaviour should be put behind a config option until v3.

JFrog Frogbot version

2.24.1

Package manager info

pip 24.0, with setup.py

Git provider

GitHub

JFrog Frogbot configuration yaml file

jfrog.yaml

name: JFrog scan

on:
  push:
    branches:
      - master
  pull_request_target:
    types: [opened, synchronize]
    branches:
      - master
  schedule:
    - cron: "0 0 * * 1-5"

concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true

jobs:
  scan:
    runs-on: <omitted>
    timeout-minutes: 30

    permissions:
      id-token: write
      pull-requests: write
      security-events: write
      contents: read

    steps:
      - uses: actions/checkout@v4

      - name: Check for Python files
        id: python
        run: |
          if [ -f ".github/jfrog/.python-version" ]; then
            echo "Found .github/jfrog/.python-version"
            python_version=$(cat .github/jfrog/.python-version)
          elif find . -name "*.py" 2>/dev/null | grep -q .; then
            echo "Python files found"
            sub_requirements=$(find . -path "**/requirements.txt" -not -path "./requirements.txt" | head -n 1)

            if [ -f "pyproject.toml" ]; then
              echo "Found pyproject.toml"
              python_version=$(grep -E '^python' pyproject.toml || echo "No version found in pyproject.toml")
            elif [ -f "requirements.txt" ]; then
              echo "Found requirements.txt"
              python_version=$(grep -oP 'This file is autogenerated by pip-compile with [Pp]ython \K[0-9]+\.[0-9]+' requirements.txt || echo '')

              if [ -z "$python_version" ]; then
                python_version="3.11"
              fi
            elif [ -n "$sub_requirements" ]; then
              echo "Found requirements.txt in subdirectory: $sub_requirements"
              python_version=$(grep -oP 'This file is autogenerated by pip-compile with [Pp]ython \K[0-9]+\.[0-9]+' "$sub_requirements" || echo '')

              if [ -z "$python_version" ]; then
                python_version="3.11"
              fi
            elif [ -f ".python-version" ]; then
              python_version=$(cat .python-version)
            else
              python_version="Unknown"
            fi
            echo "Python version: $python_version"
          else
            python_version="Unknown"
            echo "No Python files found"
          fi

          echo "python_version=$python_version" >> $GITHUB_OUTPUT

      - name: Check for Node.js files
        id: node
        run: |
          package_json_path=$(find . -name "package.json" | head -n 1)
          sub_json_path=$(find . -name "package.json" -not -path "./package.json" | head -n 1)

          if [ -n "$package_json_path" ]; then
            echo "Node.js project found at $package_json_path"
            node_found="true"
          elif [ -n "$sub_json_path" ]; then
            echo "Node.js project found at $sub_json_path"
            node_found="true"
          else
            node_found="false"
            echo "No Node.js files found"
          fi

          echo "node_found=$node_found" >> $GITHUB_OUTPUT

      - uses: actions/setup-python@v5
        if: ${{ steps.python.outputs.python_version != 'Unknown' }}
        with:
          python-version: ${{ steps.python.outputs.python_version }}

      - uses: actions/setup-node@v4
        if: ${{ steps.node.outputs.node_found == 'true' }}
        with:
          node-version: 22

      - name: Install Yarn
        if: ${{ steps.node.outputs.node_found == 'true' }}
        run: |
          npm install --global yarn

      - name: Set JFrog base branch if master
        if: github.event_name == 'schedule' || github.event_name == 'push'
        run: echo "JF_GIT_BASE_BRANCH=master" >> $GITHUB_ENV

      - name: Installation
        if: ${{ hashFiles('.github/jfrog/pre_install.sh') != '' }}
        run: |
          ./.github/jfrog/pre_install.sh

      - name: Read Frogbot path exclusions
        if: ${{ hashFiles('.github/jfrog/path_exclusions.txt') != '' }}
        run: |
          EXCLUSIONS="*git*;*node_modules*;*target*;*venv*;*test*;$(cat .github/jfrog/path_exclusions.txt)"
          echo "JF_PATH_EXCLUSIONS=$EXCLUSIONS" >> $GITHUB_ENV

      - uses: jfrog/frogbot@v2
        with:
          oidc-provider-name: github
          oidc-audience: jfrog-github
        env:
          JF_DEPS_REPO: pypi-virtual
          JF_GIT_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          JF_MIN_SEVERITY: Critical
          JF_URL: <omitted>

        # The Frogbot success comment is just a big, froggy banner advertising this tool--which we already pay for.
      - name: Delete Frogbot success comment
        if: github.event_name == 'pull_request_target'
        uses: actions/github-script@v7
        with:
          script: |
            const prNumber = context.payload.pull_request.number;
            const comments = await github.rest.issues.listComments({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: prNumber
            });

            for (const comment of comments.data) {
              if (comment.body.includes("Frogbot scanned this pull request and did not find any new security issues")) {
                await github.rest.issues.deleteComment({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  comment_id: comment.id
                });
                console.log(`Deleted comment ID: ${comment.id} (${context.repo.owner}/${context.repo.repo})`);
              }
            }

Operating system type and version

Ubuntu 22.04.5 LTS

JFrog Xray version

?

[attiasas]

Hi @williamvigolo-bmll, thank you for reporting this issue.
It is probably because of this PR: jfrog/jfrog-cli-security#255
You can revert to the old behavior by passing: JFROG_CLI_CLEAN_SUB_MODULES=TRUE or adjusting your configurations
(--working-dir flag / Frogbot config...)

[williamvigolo-bmll]

Thanks @attiasas, this looks like the culprit. We'll try out your workaround until we get around to sorting out the subpackages so they can build for Frogbot's purposes

[thomasheslinbmll]

@attiasas can you advise how to handle this elegantly in our GitHub Org? Since the workflow in each repo uses pull_request_target we are having to merge PRs with Frogbot errors.

[attiasas]

@thomasheslinbmll,
Adding the env var should revert to the old behavior.
The automatic detection should detect the submodules in the repository, if you want to scan only subset of them (or the detection is wrong at your case) you can define the actual targets you want to scan using flag --working-dir or using Frogbot config to configure the "projects"

[thomasheslinbmll]

Thanks @attiasas. Wouldn't this require pushing that env change to master (given the workflow is triggered by pull_request_target)? Is there a recommended way of testing these changes?

Also, as @williamvigolo-bmll pointed out, we updated both the action ref (uses: jfrog/[email protected]) as well as the input (version: "2.24.0"). Are both required & recommended for pinning to this version? Would be ideal to have a bit of documentation as to the differences.

[thomasheslinbmll]

@attiasas ?