Dockercode: implementation progress and challenges

[ba2512005]

I updated the dockerfile to be able to run semi properly, but am getting issues with
the setup.py file in llollms_core:

[lollms-webui builder 6/10] RUN git clone --depth 1 --recurse-submodules https://github.com/ParisNeo/lollms-webui.git && cd lollms-webui && conda run -n lollms_env bash -c "pip install -e .":
0.506 Cloning into 'lollms-webui'...
2.211 Submodule 'lollms_core' (https://github.com/ParisNeo/lollms.git) registered for path 'lollms_core'
2.211 Submodule 'zoos/bindings_zoo' (https://github.com/ParisNeo/lollms_bindings_zoo.git) registered for path 'zoos/bindings_zoo'
2.211 Submodule 'zoos/models_zoo' (https://github.com/ParisNeo/models_zoo.git) registered for path 'zoos/models_zoo'
2.212 Submodule 'zoos/personalities_zoo' (https://github.com/ParisNeo/lollms_personalities_zoo.git) registered for path 'zoos/personalities_zoo'
2.215 Cloning into '/app/lollms-webui/lollms_core'...
3.049 Cloning into '/app/lollms-webui/zoos/bindings_zoo'...
4.073 Cloning into '/app/lollms-webui/zoos/models_zoo'...
5.083 Cloning into '/app/lollms-webui/zoos/personalities_zoo'...
11.02 Submodule path 'lollms_core': checked out '4d2e198d076f0522e107b30b025be175811df236'
11.10 Submodule path 'zoos/bindings_zoo': checked out '2411827109f040abf842285ff021e5cd5bd175df'
11.16 Submodule path 'zoos/models_zoo': checked out '8b6cde5abc82e9950ce812358371c804f0ac082d'
12.62 Submodule path 'zoos/personalities_zoo': checked out 'af62c30de8e050d06c8b0086678ec8d6e77a5218'
14.83 error: subprocess-exited-with-error
14.83
14.83 × python setup.py egg_info did not run successfully.
14.83 │ exit code: 1
14.83 ╰─> [1 lines of output]
14.83 error in Lollms-webui setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.
14.83 [end of output]

Dockerfile:
'

# Use a multi-stage build to reduce image size and improve security
FROM python:3.11-slim AS builder

# Install system dependencies for building
RUN apt-get update && apt-get install -y \
    git \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Install Miniconda
RUN curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
    && bash Miniconda3-latest-Linux-x86_64.sh -b -p /opt/conda \
    && rm Miniconda3-latest-Linux-x86_64.sh

# Add Conda to PATH
ENV PATH $PATH:/opt/conda/bin

# Create and activate Conda environment
RUN conda create --name lollms_env python=3.11 git pip -y

# Clone the repository
WORKDIR /app
RUN git clone --depth 1 --recurse-submodules https://github.com/ParisNeo/lollms-webui.git \
    && cd lollms-webui \
    && conda run -n lollms_env bash -c "pip install -e ."

# Install project dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy the rest of the application code
COPY . .

# Build-time optimizations to reduce image size
RUN find /app -type f | xargs grep -oE '\n\s+$' | sed 's/^/rm /' | bash \
    && rm -rf /var/lib/apt/lists/* \
    && conda clean -a

# Final stage: production-ready environment
FROM python:3.11-slim

# Set working directory and copy application code
WORKDIR /app
COPY --from=builder /app/lollms-webui .

# Expose port 9600
EXPOSE 9600

# Set default command to run the application
CMD ["python", "app.py"]

# # Use an official Python runtime as a parent image
# FROM python:3.11-slim
#
# # Set the working directory in the container
# WORKDIR /app
#
# # Install system dependencies
# RUN apt-get update && apt-get install -y \
#     git \
#     curl \
#     && rm -rf /var/lib/apt/lists/*
#
# # Install Miniconda
# RUN curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
#     && bash Miniconda3-latest-Linux-x86_64.sh -b -p /opt/conda \
#     && rm Miniconda3-latest-Linux-x86_64.sh
#
# # Add Conda to PATH
# ENV PATH /opt/conda/bin:$PATH
#
# # Create and activate Conda environment
# RUN conda create --name lollms_env python=3.11 git pip -y
# SHELL ["conda", "run", "-n", "lollms_env", "/bin/bash", "-c"]
#
# # Clone the repository
# RUN git clone --depth 1 --recurse-submodules https://github.com/ParisNeo/lollms-webui.git \
#     && cd lollms-webui/lollms_core \
#     && pip install -e . \
#     && cd ../.. \
#     && cd lollms-webui/utilities/pipmaster \
#     && pip install -e . \
#     && cd ../..
#
# # Install project dependencies
# WORKDIR /app/lollms-webui
# COPY requirements.txt .
# RUN pip install -r requirements.txt
#
# # Copy the rest of the application code
# COPY . .
#
# # Expose port 9600
# EXPOSE 9600
#
# # Set the default command to run the application
# CMD ["python", "app.py"]'

I've made edits to the setup.py file here to fix the error:

from pathlib import Path
from typing import Union

import setuptools

with open("README.md", "r") as fh:
    long_description = fh.read()


def read_requirements(path: Union[str, Path]):
    with open(path, "r") as file:
        return file.read().splitlines()


requirements = list(filter(None, read_requirements("requirements.txt")))
requirements_dev = list(filter(None, read_requirements("requirements_dev.txt")))
def get_all_files(path):
    path = Path(path)
    file_list = []
    for file_path in path.rglob('*'):
        if file_path.is_file():
            if file_path.name != "__pycache__" and file_path.suffix !=".pyc" and  file_path.name!="local_config.yaml" and file_path.name!=".installed" and file_path.name!=".git" and file_path.name!=".gitignore":
                file_list.append("/".join(str(file_path).replace("\\","/").split("/")[1:]))
    return file_list

setuptools.setup(
    name="lollms",
    version="10.1.0",
    author="Saifeddine ALOUI (ParisNeo)",
    author_email="[email protected]",
    description="A python library for AI personality definition",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://github.com/ParisNeo/lollms",
    packages=setuptools.find_packages(),  
    include_package_data=True,
    install_requires=requirements,
    entry_points={
        'console_scripts': [
            'lollms-elf = lollms.server.elf:main',
        ],
    },
    extras_require={"dev": requirements_dev},
    classifiers=[
        "Programming Language :: Python :: 3.11",
        "License :: OSI Approved :: Apache Software License",
        "Operating System :: OS Independent",
    ],
)

[ba2512005]

PR: #583

@ParisNeo when you have some time can you please look into this PR i submitted for fixing docker setup