-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
examples: Add test run for lv2v #283
Changes from 20 commits
1a7c557
2225784
1b3ed77
5835437
347722e
3c6489d
47feacb
3c2def5
4810532
fc26de1
9d47065
b7e4a39
6cfaf4f
aaf8338
0d23630
81da469
6c51f8e
0ffc438
5b087ef
d89d4e6
a3217a3
bc50ddb
eb14c69
e7fc022
e5123b6
88bd756
dfd732e
a5c973e
c2f76e0
afe82dc
38a5e4a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
name: FPS check for live-video-to-video pipelines | ||
|
||
on: | ||
pull_request: | ||
paths: | ||
- "runner/**" | ||
- "!runner/.devcontainer/**" | ||
push: | ||
branches: | ||
- main | ||
tags: | ||
- '*' | ||
paths: | ||
- "runner/**" | ||
- "!runner/.devcontainer/**" | ||
workflow_dispatch: | ||
|
||
jobs: | ||
run-fps-test: | ||
runs-on: ubuntu-latest | ||
strategy: | ||
max-parallel: 1 | ||
matrix: | ||
model_config: | ||
- { id: noop, input_fps: 30, exp_output_fps: 28 } | ||
- { id: liveportrait, input_fps: 30, exp_output_fps: 28 } | ||
- { id: streamdiffusion, input_fps: 30, exp_output_fps: 28 } | ||
- { id: comfyui, input_fps: 30, exp_output_fps: 28 } # only depth anything for now | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
|
||
- name: Set up Go | ||
uses: actions/setup-go@v5 | ||
with: | ||
go-version: '1.23.3' | ||
|
||
- name: Build Docker images | ||
env: | ||
MODEL_ID: ${{ matrix.model_config.id }} | ||
run: | | ||
cd runner | ||
docker build -t livepeer/ai-runner:live-base -f docker/Dockerfile.live-base . | ||
if [ "${MODEL_ID}" = "noop" ]; then | ||
docker build -t livepeer/ai-runner:live-app-noop -f docker/Dockerfile.live-app-noop . | ||
else | ||
docker build -t livepeer/ai-runner:live-base-${MODEL_ID} -f docker/Dockerfile.live-base-${MODEL_ID} . | ||
docker build -t livepeer/ai-runner:live-app-${MODEL_ID} -f docker/Dockerfile.live-app__PIPELINE__ --build-arg PIPELINE=${MODEL_ID} . | ||
fi | ||
cd .. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm I think this will make the test take waaaay longer like 30 minutes. Some ideas:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Yeah we do that on the docker build workflow. But it's kinda of a pain TBH, pretty complex, and it hurts even more to repeat all of it here 💀 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. with the self-hosted runners, we clean up disk frequently, so that we dont run out of disk space for other jobs. this means the intermediate layers etc. are also gone. as victor mentioned above (and i mentioned in the discord thread), best would be to trigger this workflow after docker build has finished and pull those images for testing. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @hjpotter92 can we not wipe the models dir if possible? that would save a lot of time, can just link the persistent path to the req path |
||
|
||
- name: Install packages | ||
run: | | ||
sudo sh -c "apt-get update && apt-get install -y libzmq3-dev && rm -rf /var/lib/apt/lists/*" | ||
|
||
- name: Clean up runner dockers | ||
run: | | ||
CONTAINERS=$(docker ps -aq --filter "name=^live-video-to-video*") | ||
if [ -n "$CONTAINERS" ]; then | ||
echo "Removing containers: $CONTAINERS" | ||
echo "$CONTAINERS" | xargs -r docker rm -f | ||
else | ||
echo "No containers to remove for model ${MODEL_ID}" | ||
fi | ||
|
||
- name: Run FPS test | ||
env: | ||
MODEL_ID: ${{ matrix.model_config.id }} | ||
INPUT_FPS: ${{ matrix.model_config.input_fps }} | ||
EXP_OUTPUT_FPS: ${{ matrix.model_config.exp_output_fps }} | ||
run: | | ||
# Redirect both stdout and stderr to a file and tee to console | ||
go run cmd/examples/live-video-to-video/main.go \ | ||
-modelid=${MODEL_ID} \ | ||
-inputfps=${INPUT_FPS} \ | ||
-expoutputfps=${EXP_OUTPUT_FPS} 2>&1 | tee output.log | ||
|
||
# Use grep with the timestamp pattern to find the TEST PASSED/FAILED message | ||
if grep -q "INFO TEST PASSED!" output.log; then | ||
echo "Test for ${MODEL_ID} pipeline PASSED" | ||
exit 0 | ||
elif grep -q "INFO TEST FAILED!" output.log; then | ||
echo "Test for ${MODEL_ID} pipeline FAILED" | ||
exit 1 | ||
else | ||
echo "Test for ${MODEL_ID} pipeline had an unknown result" | ||
echo "Full log output:" | ||
cat output.log | ||
exit 1 | ||
fi |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,6 +8,7 @@ output | |
aiModels.json | ||
models | ||
checkpoints | ||
runner/run-lv2v.log | ||
|
||
# IDE | ||
.vscode | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should use
pipeline
instead ofmodel
everywhere in this file. It is only called model on the legacy code since we reused the existingmodel_id
param to change which live pipeline to run, but ideally we keep that "misnaming" on the minimum places possible (and always with a comment disclaimer)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw for more context: https://discord.com/channels/423160867534929930/1308107200522227712/1315767248710930465