Skip to content
Optimum TPU / Test TGI on TPU / Jetstream Pytorch

fix: graceful shutdown was not working with entrypoint, exec launcher #41

Sign in to view logs

fix: graceful shutdown was not working with entrypoint, exec launcher

fix: graceful shutdown was not working with entrypoint, exec launcher #41

Workflow file for this run

.github/workflows/test-pytorch-xla-tpu-tgi-jetstream.yml at 7f40e9f
name: Optimum TPU / Test TGI on TPU / Jetstream Pytorch
on:
pull_request:
branches: [ main ]
paths:
- "text-generation-inference/**"
# This can be used to trigger workflow from the web interface
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
do-the-job:
name: Run TGI tests - Jetstream Pytorch
runs-on:
group: gcp-ct5lp-hightpu-8t
container:
image: us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:r2.4.0_3.10_tpuvm
options: --shm-size "16gb" --ipc host --privileged ${{ vars.V5_LITEPOD_8_ENV}} -v /mnt/hf_cache:/mnt/hf_cache
env:
PJRT_DEVICE: TPU
HF_HUB_CACHE: /mnt/hf_cache/cache_huggingface
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Build and test TGI server
run: |
HF_TOKEN=${{ secrets.HF_TOKEN_OPTIMUM_TPU_CI }} make tgi_test_jetstream