dvc - 10427 Clean Environment Sandbox

[nablabits]

Related to:

• dvc queue status doesn't report active workers iterative/dvc#10427

This is just a clean environment where reviewers can check that the approach works as it takes some time to config it from the scratch. You will find more context in this comment but below are the instructions to reproduce and verify the solution

How to Test this

• Clone the repo
• Install deps
• Checkcout the branch. Make sure the submodules dvc & dvc-task are correctly imported

Reproduce the Issue

• Queue a couple of experiments and run them

dvc exp run --queue --set-param "train.fine_tune_args.epochs=2,3" && \
dvc queue start -j 2 -v

• Give it some time to complete, you can check this with tail .dvc/tmp/exps/celery/dvc-exp-worker-1.out if something like this appears, it means that they have finished:

[2024-09-05 08:05:31,883: INFO/MainProcess] done
2024-09-05 08:05:31,885 DEBUG: Analytics is enabled.
[2024-09-05 08:05:31,885: DEBUG/MainProcess] Analytics is enabled.
2024-09-05 08:05:31,986 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmpol8kssna', '-v']
[2024-09-05 08:05:31,986: DEBUG/MainProcess] Trying to spawn ['daemon', 'analytics', '/tmp/tmpol8kssna', '-v']
2024-09-05 08:05:31,999 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmpol8kssna', '-v'] with pid 594310
[2024-09-05 08:05:31,999: DEBUG/MainProcess] Spawned ['daemon', 'analytics', '/tmp/tmpol8kssna', '-v'] with pid 594310
(example-experiments)

• Verify that a couple of files with this body remain in in:

grep 'eyJtZXRob2QiOiAic2h1dGRvd24iLCAiYXJndW1lbnRzIjoge30sICJkZXN0aW5hdGlvbiI6IG51bGwsICJwYXR0ZXJuIjogbnVsbCwgIm1hdGNoZXIiOiBudWxsfQ==' \
.dvc/tmp/exps/celery/broker/in/*

Verify the solution

• Override the default source for dvc-task that will contain the changes:

export PYTHONPATH="`pwd`/sources/dvc-task/src"

• Clean everything from previous round and start over

dvc exp clean && dvc queue remove --all && \
rm -rf .dvc/cache/runs && \
echo '' > .dvc/tmp/exps/celery/*.out && \
dvc exp run --queue --set-param "train.fine_tune_args.epochs=2,3" && \
dvc queue start -j 2 -v

• Verify that no file with this body remains in in:

grep 'eyJtZXRob2QiOiAic2h1dGRvd24iLCAiYXJndW1lbnRzIjoge30sICJkZXN0aW5hdGlvbiI6IG51bGwsICJwYXR0ZXJuIjogbnVsbCwgIm1hdGNoZXIiOiBudWxsfQ==' \
.dvc/tmp/exps/celery/broker/in/*