Moving ml docker from Pip to Poetry and updating onnx to 1.17.0 version #34529

rshunim · 2024-12-05T10:27:11Z

Status

Ready

Related Issues

fixes: link to the issue

Description

Moving the ml docker to poetry and updating onnx to 1.17.0

CLAassistant · 2024-12-05T10:27:22Z

All committers have signed the CLA.

Co-authored-by: dorschw <[email protected]>

jlevypaloalto

The switch to poetry is fantastic.
Let's link the content PR where we tested all the ML scripts that use this image.

docker/ml/pyproject.toml

Co-authored-by: Jacob Levy <[email protected]>

rshunim · 2024-12-10T08:08:53Z

The Content PR demisto/content#37612
The pre-commit passed with running the all UTs with devdemisto/ml:1.0.0.118104

xsoar-bot · 2024-12-11T12:26:09Z

Docker Image Ready - Dev

Docker automatic build has deployed your docker image: devdemisto/ml:1.0.0.1790385
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/ml/tags
Get started by pulling the image:

docker pull devdemisto/ml:1.0.0.1790385

Docker Metadata

Image Size: 1004.66 MB
Image ID: sha256:b1c823abf4b965f8e71057b8e8a4796dd9c824f127ad6ff346e24d467de370b1
Created: 2024-12-11T12:16:59.547371522Z
Arch: linux/amd64
Command: ["python3"]
Environment:
- PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
- LANG=C.UTF-8
- GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D
- PYTHON_VERSION=3.10.14
- PYTHON_PIP_VERSION=23.0.1
- PYTHON_SETUPTOOLS_VERSION=65.5.1
- PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py
- PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9
- DOCKER_IMAGE=devdemisto/ml:1.0.0.1790385
- HF_HOME=/ml/.cache
- NLTK_DATA=/ml/nltk_data
- MPLCONFIGDIR=/ml/matplotlib
Labels:
- org.opencontainers.image.authors:Demisto <[email protected]>
- org.opencontainers.image.revision:9018080c68370c467616326e2f95049c777ca939
- org.opencontainers.image.version:1.0.0.1790385

xsoar-bot · 2024-12-11T12:35:27Z

Docker Image Ready - Dev

Docker automatic build has deployed your docker image: devdemisto/ml:1.0.0.1790411
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/ml/tags
Get started by pulling the image:

docker pull devdemisto/ml:1.0.0.1790411

Docker Metadata

Image Size: 1004.67 MB
Image ID: sha256:3eb93e5709e678ddc1f559669f150e778d437ac62db9f5ebd350b0216d9c7d63
Created: 2024-12-11T12:25:03.014102487Z
Arch: linux/amd64
Command: ["python3"]
Environment:
- PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
- LANG=C.UTF-8
- GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D
- PYTHON_VERSION=3.10.14
- PYTHON_PIP_VERSION=23.0.1
- PYTHON_SETUPTOOLS_VERSION=65.5.1
- PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py
- PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9
- DOCKER_IMAGE=devdemisto/ml:1.0.0.1790411
- HF_HOME=/ml/.cache
- NLTK_DATA=/ml/nltk_data
- MPLCONFIGDIR=/ml/matplotlib
Labels:
- org.opencontainers.image.authors:Demisto <[email protected]>
- org.opencontainers.image.revision:606f85d5e5549bf569308c6f4a7976e5e80c13d3
- org.opencontainers.image.version:1.0.0.1790411

rshunim · 2024-12-11T13:19:14Z

Link to the Content PR demisto/content#37612
with the last ml docker devdemisto/ml:1.0.0.1790385

nltk.download('punkt_tab', download_dir='/ml/nltk_data')

xsoar-bot · 2024-12-11T14:31:14Z

Docker Image Ready - Dev

Docker automatic build has deployed your docker image: devdemisto/ml:1.0.0.1791382
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/ml/tags
Get started by pulling the image:

docker pull devdemisto/ml:1.0.0.1791382

Docker Metadata

Image Size: 1020.83 MB
Image ID: sha256:86ecb0ad556077d35707f05d4826a79fee55a2b23159c51c867deab63a792334
Created: 2024-12-11T14:22:01.093863451Z
Arch: linux/amd64
Command: ["python3"]
Environment:
- PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
- LANG=C.UTF-8
- GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D
- PYTHON_VERSION=3.10.14
- PYTHON_PIP_VERSION=23.0.1
- PYTHON_SETUPTOOLS_VERSION=65.5.1
- PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py
- PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9
- DOCKER_IMAGE=devdemisto/ml:1.0.0.1791382
- HF_HOME=/ml/.cache
- NLTK_DATA=/ml/nltk_data
- MPLCONFIGDIR=/ml/matplotlib
Labels:
- org.opencontainers.image.authors:Demisto <[email protected]>
- org.opencontainers.image.revision:b6e9e99d3baf51ac5c80e8100cca95a8350a0650
- org.opencontainers.image.version:1.0.0.1791382

rshunim · 2024-12-23T13:04:35Z

last link to the Content PR
demisto/content#37612

xsoar-bot · 2024-12-24T09:54:30Z

Docker Image Ready - Dev

Docker automatic build has deployed your docker image: devdemisto/ml:1.0.0.1862390
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/ml/tags
Get started by pulling the image:

docker pull devdemisto/ml:1.0.0.1862390

Docker Metadata

Image Size: 1045.15 MB
Image ID: sha256:328154ebece9187a933974c903d6a517a0f17d50e4f58cbf17074d11849f9bb1
Created: 2024-12-24T09:46:49.224134Z
Arch: linux/amd64
Command: ["python3"]
Environment:
- PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
- LANG=C.UTF-8
- GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D
- PYTHON_VERSION=3.10.14
- PYTHON_PIP_VERSION=23.0.1
- PYTHON_SETUPTOOLS_VERSION=65.5.1
- PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py
- PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9
- DOCKER_IMAGE=devdemisto/ml:1.0.0.1862390
- HF_HOME=/ml/.cache
- NLTK_DATA=/ml/nltk_data
- MPLCONFIGDIR=/ml/matplotlib
Labels:
- org.opencontainers.image.authors:Demisto <[email protected]>
- org.opencontainers.image.revision:f5335ef9edf8696625fe36346f8c26d8509d97c3
- org.opencontainers.image.version:1.0.0.1862390

rshunim · 2024-12-25T11:47:07Z

https://gitlab.xdr.pan.local/xdr/cortex-content/content/-/pipelines/1868975

xsoar-bot · 2024-12-25T12:22:34Z

Docker Image Ready - Production

Docker automatic build has deployed your docker image: demisto/ml:1.0.0.1870375
It is available now on docker hub at: https://hub.docker.com/r/demisto/ml/tags
Get started by pulling the image:

docker pull demisto/ml:1.0.0.1870375

Docker Metadata

Image Size: 1045.16 MB
Image ID: sha256:4911ae488aa28c48b25cfe88f768270ac5923c0d8c6e47fe5a5353d386b87f00
Created: 2024-12-25T12:11:36.35359756Z
Arch: linux/amd64
Command: ["python3"]
Environment:
- PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
- LANG=C.UTF-8
- GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D
- PYTHON_VERSION=3.10.14
- PYTHON_PIP_VERSION=23.0.1
- PYTHON_SETUPTOOLS_VERSION=65.5.1
- PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py
- PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9
- DOCKER_IMAGE=demisto/ml:1.0.0.1870375
- HF_HOME=/ml/.cache
- NLTK_DATA=/ml/nltk_data
- MPLCONFIGDIR=/ml/matplotlib
Labels:
- org.opencontainers.image.authors:Demisto <[email protected]>
- org.opencontainers.image.revision:ad3d3c63663e51e66b47251d677b7995cfd5e71f
- org.opencontainers.image.version:1.0.0.1870375

remove pytorch and lock

3c5576b

rshunim requested a review from dorschw December 5, 2024 10:27

rshunim self-assigned this Dec 5, 2024

rshunim requested a review from jlevypaloalto as a code owner December 5, 2024 10:27

rshunim removed the request for review from jlevypaloalto December 5, 2024 10:27

rshunim and others added 24 commits December 5, 2024 12:27

Merge branch 'master' into onnx_version

eb8dbd2

remove Pipfile and Pipfile.lock

b9a1ade

Merge remote-tracking branch 'origin/onnx_version' into onnx_version

3ce324a

requirements.txt .

dc712e6

test

be76396

test

2b9efad

test

6b425a4

Merge branch 'master' into onnx_version

242efd7

revert

6f3fdff

Merge branch 'master' into onnx_version

03ac77e

AS

f265d08

Merge remote-tracking branch 'origin/onnx_version' into onnx_version

fe21607

revert to master

c8e43b9

split lines

2291718

test

74622eb

test

3987b07

test

02a58ff

Merge branch 'master' into onnx_version

cbdc4ba

copy from mlurlphishing

cc8d6a6

separate the lines

e3a44f6

test

f473bd0

revert and split

f6d0e9f

test

a8aa8bd

test

4f628ea

rshunim and others added 3 commits December 9, 2024 15:43

Update docker/ml/pyproject.toml

c0b6f67

Co-authored-by: dorschw <[email protected]>

CR

e46419e

Merge branch 'master' into onnx_version

06b9122

rshunim changed the title ~~remove pytorch and lock~~ Moving ml docker from Pip to Poetry and updating onnx to 1.17.0 version Dec 9, 2024

dorschw approved these changes Dec 9, 2024

View reviewed changes

CR

f72f8cc

jlevypaloalto reviewed Dec 9, 2024

View reviewed changes

docker/ml/pyproject.toml Outdated Show resolved Hide resolved

Update docker/ml/pyproject.toml

396e633

Co-authored-by: Jacob Levy <[email protected]>

rshunim and others added 2 commits December 11, 2024 13:47

Merge branch 'master' into onnx_version

9018080

Empty-Commit

606f85d

add

b6e9e99

nltk.download('punkt_tab', download_dir='/ml/nltk_data')

Merge branch 'master' into onnx_version

9bf14ec

rshunim and others added 3 commits December 23, 2024 16:38

Merge branch 'master' into onnx_version

188a113

add annotated-types to known_licenses.json

6cf6831

Merge remote-tracking branch 'origin/onnx_version' into onnx_version

237d24c

rshunim requested a review from anara123 as a code owner December 24, 2024 08:09

rshunim and others added 3 commits December 24, 2024 10:10

Merge branch 'master' into onnx_version

bc13489

new poetry.lock

19375a4

add attrs to the known_licenses.json

f5335ef

rshunim merged commit ad3d3c6 into master Dec 25, 2024
5 of 6 checks passed

rshunim deleted the onnx_version branch December 25, 2024 11:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Moving ml docker from Pip to Poetry and updating onnx to 1.17.0 version #34529

Moving ml docker from Pip to Poetry and updating onnx to 1.17.0 version #34529

rshunim commented Dec 5, 2024 •

edited

Loading

CLAassistant commented Dec 5, 2024 •

edited

Loading

jlevypaloalto left a comment

rshunim commented Dec 10, 2024

xsoar-bot commented Dec 11, 2024

xsoar-bot commented Dec 11, 2024

rshunim commented Dec 11, 2024 •

edited

Loading

xsoar-bot commented Dec 11, 2024

rshunim commented Dec 23, 2024

xsoar-bot commented Dec 24, 2024

rshunim commented Dec 25, 2024

xsoar-bot commented Dec 25, 2024

Moving ml docker from Pip to Poetry and updating onnx to 1.17.0 version #34529

Moving ml docker from Pip to Poetry and updating onnx to 1.17.0 version #34529

Conversation

rshunim commented Dec 5, 2024 • edited Loading

Status

Related Issues

Description

CLAassistant commented Dec 5, 2024 • edited Loading

jlevypaloalto left a comment

Choose a reason for hiding this comment

rshunim commented Dec 10, 2024

xsoar-bot commented Dec 11, 2024

Docker Image Ready - Dev

Docker Metadata

xsoar-bot commented Dec 11, 2024

Docker Image Ready - Dev

Docker Metadata

rshunim commented Dec 11, 2024 • edited Loading

xsoar-bot commented Dec 11, 2024

Docker Image Ready - Dev

Docker Metadata

rshunim commented Dec 23, 2024

xsoar-bot commented Dec 24, 2024

Docker Image Ready - Dev

Docker Metadata

rshunim commented Dec 25, 2024

xsoar-bot commented Dec 25, 2024

Docker Image Ready - Production

Docker Metadata

rshunim commented Dec 5, 2024 •

edited

Loading

CLAassistant commented Dec 5, 2024 •

edited

Loading

rshunim commented Dec 11, 2024 •

edited

Loading