HTTP 403 Forbidden Error During Model Prediction Due to File Access Denial when using with Cloud Storage Sync

[SabolSocare]

I am unable to use the Yolo ML Backend model to annotate image data from Cloud Storage.

/label-studio/media/upload, upload_dir=/Users/admin/Library/Application Support/label-studio/media/upload
[2024-11-04 16:16:50,467] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::get_local_path::144] Resolving url using hostname [http://localhost:8089/]: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg
[2024-11-04 16:16:50,468] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::download_and_cache::205] Download http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg to /Users/admin/Library/Caches/label-studio/24791863__1f742291bf201cfc4c95d5ab7ce3bfc5.jpg
[2024-11-04 16:16:50,468] [DEBUG] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::download_and_cache::218] Authorization token is used for download_and_cache
[2024-11-04 16:16:50,586] [ERROR] [label_studio_ml.exceptions::exception_f::53] Traceback (most recent call last):
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py", line 69, in _predict
    response = model.predict(tasks, context=context, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py", line 120, in predict
    path = model.get_path(task)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py", line 187, in get_path
    else get_local_path(task_path, task_id=task.get("id"))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 166, in get_local_path
    filepath = download_and_cache(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 221, in download_and_cache
    r.raise_for_status()
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg

Traceback (most recent call last):
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py", line 69, in _predict
    response = model.predict(tasks, context=context, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py", line 120, in predict
    path = model.get_path(task)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py", line 187, in get_path
    else get_local_path(task_path, task_id=task.get("id"))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 166, in get_local_path
    filepath = download_and_cache(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 221, in download_and_cache
    r.raise_for_status()
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg

[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 500 INTERNAL SERVER ERROR
[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json
Content-Length: 2389

[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{\n  "detail": "HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg",\n  "request": {},\n  "result": {\n    "traceback": "Traceback (most recent call last):\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py\\", line 39, in exception_f\\n    return f(*args, **kwargs)\\n           ^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py\\", line 69, in _predict\\n    response = model.predict(tasks, context=context, **params)\\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py\\", line 120, in predict\\n    path = model.get_path(task)\\n           ^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py\\", line 187, in get_path\\n    else get_local_path(task_path, task_id=task.get(\\"id\\"))\\n         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py\\", line 166, in get_local_path\\n    filepath = download_and_cache(\\n               ^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py\\", line 221, in download_and_cache\\n    r.raise_for_status()\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py\\", line 1024, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg\\n"\n  },\n  "status": 500\n}\n'
[2024-11-04 16:16:50,589] [INFO] [werkzeug::_log::97] 127.0.0.1 - - [04/Nov/2024 16:16:50] "POST /predict HTTP/1.1" 500 -

[makseq]

How does your docker-compose.yml look?
Did you follow step #1 from https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/README.md#quick-start ?

[SabolSocare]

Here's my Docker Compose configuration for the YOLO service:

It works when the image is uploaded from the browser, but it does not work with images synced from cloud storage.

version: "3.8"

services:
  yolo:
    container_name: yolo
    image: humansignal/yolo:v0
    build:
      context: .
      args:
        TEST_ENV: ${TEST_ENV}
    environment:
      # Optional basic auth for the model server
      - BASIC_AUTH_USER=
      - BASIC_AUTH_PASS=
      # Logging and server configuration
      - LOG_LEVEL=DEBUG
      - WORKERS=1
      - THREADS=8
      - MODEL_DIR=/data/models
      - PYTHONPATH=/app

      # Label Studio integration
      - LABEL_STUDIO_URL=http://127.0.0.1:8089
      - LABEL_STUDIO_API_KEY=9e06b3eca8f7edfc0f9b9XXXXXXXXXX

      # YOLO-specific configuration
      - ALLOW_CUSTOM_MODEL_PATH=true
      - DEBUG_PLOT=false
      - MODEL_SCORE_THRESHOLD=0.5
      - MODEL_ROOT=/app/models
    ports:
      - "9090:9090"
    volumes:
      - "./data/server:/data"
      - "./models:/app/models"
      - "./cache_dir:/app/cache_dir"