Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Action to Compare Dependencies #29728

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
3a43cd2
added gitub action
shmuel44 Jul 16, 2024
c0ae7c1
pipenv
shmuel44 Jul 16, 2024
8290786
pipenv
shmuel44 Jul 16, 2024
69aee8e
pipenv
shmuel44 Jul 16, 2024
2c249f8
pipenv
shmuel44 Jul 16, 2024
debe4a9
remove pipenv shell
shmuel44 Jul 16, 2024
4956cbb
pipenv install
shmuel44 Jul 16, 2024
df88ee6
path
shmuel44 Jul 16, 2024
76fba50
run python
shmuel44 Jul 16, 2024
7faa2e7
pipenv install
shmuel44 Jul 16, 2024
ce78770
remove changed-files
shmuel44 Jul 16, 2024
0b0e2df
error
shmuel44 Jul 16, 2024
a73f7d5
return 1/0
shmuel44 Jul 16, 2024
3ac5966
main
shmuel44 Jul 16, 2024
ce86ffb
no return
shmuel44 Jul 16, 2024
e059f9f
return
shmuel44 Jul 16, 2024
b78da47
print
shmuel44 Jul 16, 2024
9e87de4
fix
shmuel44 Jul 16, 2024
39dd6ac
line number
shmuel44 Jul 16, 2024
942f0d1
cd
shmuel44 Jul 16, 2024
0153814
print
shmuel44 Jul 16, 2024
31a9ec3
line_number
shmuel44 Jul 16, 2024
ed62dfb
str
shmuel44 Jul 16, 2024
36a9ada
Update .github/workflows/sync_native_dependency.yml
shmuel44 Jul 18, 2024
19fbf10
Update .github/workflows/sync_native_dependency.yml
shmuel44 Jul 18, 2024
8286c18
Update utils/compare_dependency_constraints.py
shmuel44 Jul 18, 2024
70f6039
Update utils/compare_dependency_constraints.py
shmuel44 Jul 18, 2024
622511e
CR
shmuel44 Jul 18, 2024
5268bcd
CR
shmuel44 Jul 18, 2024
3b5110e
pull from master
shmuel44 Aug 12, 2024
3d87c1b
compares the dependencies of `python3-ubi` against `py3-tools-ubi`
shmuel44 Aug 12, 2024
1a0111c
Update docker/py3-native/pyproject.toml
shmuel44 Aug 12, 2024
79e9bb8
.get(dependency)
shmuel44 Aug 12, 2024
c5762b2
Merge branch 'sk_github_sync_native' of https://github.com/demisto/do…
shmuel44 Aug 12, 2024
30267da
python3-ubi > py3-tools
shmuel44 Aug 12, 2024
68467b9
reference_image=PY3_TOOLS_IMAGE,
shmuel44 Aug 12, 2024
6b47d36
NATIVE_IMAGE
shmuel44 Aug 12, 2024
9e48b25
ruff
shmuel44 Aug 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/sync_native_dependency.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Compare Dependency Constraints
on:
pull_request:
branches:
- master
jobs:
compare_dependency_constraints_script:
runs-on: ubuntu-latest
if: github.repository == 'demisto/dockerFiles'
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.11
uses: actions/setup-python@v3
with:
python-version: "3.11"
- name: Install dependencies with pipenv
run: |
python -m pip install --upgrade pip
pip install pipenv
pipenv install
- name: Run compare_dependency_constraints Script.
run: |
ls
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
echo "==== $(date): Starting environment setup... ===="
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
cd "$GITHUB_WORKSPACE"
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
echo "==== $(date): Running chceking Native dependency ===="
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
pipenv run python utils/compare_dependency_constraints.py
2 changes: 1 addition & 1 deletion docker/py3-native/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ parse-emails = "*"
slack-sdk = {extras = ["optional"], version = "*"}
pyjwt = {extras = ["crypto"],version = "*"}
google-api-python-client = "*"
exchangelib = ">=5.4.1"
exchangelib = ">=5.4.0"
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
requests-oauthlib = "*"
oauthlib = {extras = ["signedtoken"], version = "*"}
requests-ntlm = "==1.1.0"
Expand Down
40 changes: 34 additions & 6 deletions utils/compare_dependency_constraints.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from typing import Any, NamedTuple
import requests
import toml
import sys

DOCKER_FOLDER = Path(__file__).parent.parent / "docker"
NATIVE_IMAGE = "py3-native"
Expand Down Expand Up @@ -49,8 +50,25 @@ class Discrepancy(NamedTuple):
in_native: str | None = None

def __str__(self) -> str:
return f"{self.dependency}: {self.in_image or 'missing'} in {self.image}, {self.in_native or 'missing'} in native"
return f"{self.dependency}: {self.in_image or 'missing'} in {self.image} image, {self.in_native or 'missing'} in Native image"

def find_library_line_number(lib_name: str, file_path: Path):
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
"""
Searches for a library in the pyproject.toml or Pipfile file and returns the line number where it is found.

Parameters:
- lib_name: The name of the library to search for.
- file_path: The directory containing the pyproject.toml or Pipfile.

Returns:
- The line number containing the library name, or None if the library is not found.
"""
with file_path.open('r') as file:
for line_number, line in enumerate(file, start=1): # Start counting from line 1
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
if lib_name in line:
return line_number

return None

def compare_constraints(images_contained_in_native: list[str]):
native_constraints = (
Expand All @@ -59,9 +77,8 @@ def compare_constraints(images_contained_in_native: list[str]):
| parse_constraints(NATIVE_IMAGE)
)
native_constraint_keys = set(native_constraints.keys())

discrepancies: list[Discrepancy] = []
for image in images_contained_in_native:
discrepancies: list[Discrepancy] = []

constraints = parse_constraints(image)
constraint_keys = set(constraints.keys())
Expand Down Expand Up @@ -93,8 +110,18 @@ def compare_constraints(images_contained_in_native: list[str]):
)
)

for discrepancy in discrepancies:
print(str(discrepancy))
for discrepancy in discrepancies:
file_pyproject_path = DOCKER_FOLDER / f"{discrepancy.image}/pyproject.toml"
if file_pyproject_path.exists():
file_path = file_pyproject_path
else:
file_path = DOCKER_FOLDER / f"{discrepancy.image}/Pipfile"
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
line_number = find_library_line_number(discrepancy.dependency, file_path)
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved

print(
f"::error file={file_path},line={line_number},endLine={line_number},title=Native Image Discrepancy::{discrepancy}"
shmuel44 marked this conversation as resolved.
Show resolved Hide resolved
)
return int(bool(discrepancies))


def load_native_image_conf() -> list[str]:
Expand All @@ -106,4 +133,5 @@ def load_native_image_conf() -> list[str]:
)["native_images"]["native:candidate"]["supported_docker_images"]


compare_constraints(load_native_image_conf())
if __name__ == "__main__":
sys.exit(compare_constraints(load_native_image_conf()))
Loading