Skip to content
This repository has been archived by the owner on Jul 10, 2024. It is now read-only.

Commit

Permalink
SUBMARINE-1365. Fix failure python unit test environment
Browse files Browse the repository at this point in the history
### What is this PR for?
https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929
we got following error in python unit test.
`[24|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:25]Collecting wheel
[25|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:26] Downloading wheel-0.38.4-py3-none-any.whl (36 kB)
[26|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:27]Installing collected packages: wheel, setuptools
[27|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:28] Attempting uninstall: setuptools
[28|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:29] Found existing installation: setuptools 65.5.0
[29|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:30] Uninstalling setuptools-65.5.0:
[30|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:31] Successfully uninstalled setuptools-65.5.0
[31|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:32]Successfully installed setuptools-66.0.0 wheel-0.38.4
[32|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:33]Obtaining file:///home/runner/work/submarine/submarine/submarine-sdk/pysubmarine
[33|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:34] Preparing metadata (setup.py): started
[34|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:35] Preparing metadata (setup.py): finished with status 'error'
[35|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:36] error: subprocess-exited-with-error
[36|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:37]
[37|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:38] × python setup.py egg_info did not run successfully.
[38|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:39] │ exit code: 1
[39|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:40] ╰─> [27 lines of output]
[40|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:41] /opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/dist.py:543: UserWarning: The version specified ('0.8.0-SNAPSHOT') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
[41|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:42] warnings.warn(
[42|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:43] running egg_info
[43|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:44] Traceback (most recent call last):
[44|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:45] File "<string>", line 2, in <module>
[45|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:46] File "<pip-setuptools-caller>", line 34, in <module>
[46|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:47] File "/home/runner/work/submarine/submarine/submarine-sdk/pysubmarine/setup.py", line 21, in <module>
[47|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:48] setup(
[48|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:49] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
[49|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:50] return distutils.core.setup(**attrs)
[50|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:51] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
[51|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:52] return run_commands(dist)
[52|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:53] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
[53|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:54] dist.run_commands()
[54|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:55] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
[55|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:56] self.run_command(cmd)
[56|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:57] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/dist.py", line 1208, in run_command
[57|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:58] super().run_command(command)
[58|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:59] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
[59|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:60] cmd_obj.ensure_finalized()
[60|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:61] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
[61|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:62] self.finalize_options()
[62|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:63] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 219, in finalize_options
[63|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:64] parsed_version = parse_version(self.egg_version)
[64|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:65] File "/opt/hostedtoolcache/Python/3.10.9/x64/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__
[65|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:66] raise InvalidVersion(f"Invalid version: '\{version}'")
[66|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:67] pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '0.8.0-SNAPSHOT'
[67|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:68] [end of output]
[68|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:69]
[69|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:70] note: This error originates from a subprocess, and is likely not a problem with pip.
[70|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:71]error: metadata-generation-failed
[71|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:72]
[72|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:73]× Encountered error while generating package metadata.
[73|https://github.com/apache/submarine/actions/runs/3929247568/jobs/6717831929#step:15:74]╰─> See above for output.`
I found a similar issue and it looks like there's some trouble on setuptools.

pypa/setuptools#3772

I'd like to take their adoption to fix the setuptools to 65.7.0 as a workaround here.
### What type of PR is it?
Hot fix

### Todos
N/A

### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1365
### How should this be tested?
should run python unit test as expected.
### Screenshots (if appropriate)

### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No

Author: FatalLin <[email protected]>

Signed-off-by: Thinking <[email protected]>

Closes #1040 from FatalLin/SUBMARINE-1365 and squashes the following commits:

e2fe9b5 [FatalLin] SUBMARINE-1365. Fix faulure python unit test environment
  • Loading branch information
FatalLin authored and cdmikechen committed Jan 19, 2023
1 parent 4cd2af1 commit 793c8a2
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- uses: actions/checkout@v3
- name: Install dependencies
run: |
pip install --upgrade pip setuptools wheel
pip install --upgrade pip setuptools==65.7.0 wheel
pip install -r ./dev-support/style-check/python/lint-requirements.txt
pip install -r ./dev-support/style-check/python/mypy-requirements.txt
- name: List installed packages
Expand Down Expand Up @@ -64,7 +64,7 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install requirements
run: |
pip install --upgrade pip setuptools wheel
pip install --upgrade pip setuptools==65.7.0 wheel
pip install -r ./submarine-sdk/pysubmarine/github-actions/test-requirements.txt
- name: Install pysubmarine with tf1 and pytorch
if: ${{ matrix.tf-version == '1.15.0' }}
Expand Down Expand Up @@ -155,13 +155,13 @@ jobs:
- name: Install tf1 dependencies
if: ${{ matrix.tf-version == '1.15.0' }}
run: |
pip install --upgrade pip setuptools wheel
pip install --upgrade pip setuptools==65.7.0 wheel
pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf,pytorch]
pip install -r ./submarine-sdk/pysubmarine/github-actions/test-requirements.txt
- name: Install tf2 dependencies
if: startsWith(matrix.tf-version, '2.')
run: |
pip install --upgrade pip setuptools wheel
pip install --upgrade pip setuptools==65.7.0 wheel
pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
pip install --no-cache-dir 'tensorflow~=${{ matrix.tf-version }}'
pip install -r ./submarine-sdk/pysubmarine/github-actions/test-requirements.txt
Expand Down

0 comments on commit 793c8a2

Please sign in to comment.