e2e test takes over an hour

[RobotSail]

This needs to be way quicker. Realistically we should always be running off of a precomputed dataset.

[nathan-weinberg]

Duplicate of instructlab/instructlab#2496

[ktam3]

@RobotSail can we close this in favor of instructlab/instructlab#2496

[RobotSail]

@ktam3 We should keep this one as it's specific to the training repo and may not reflect the needs of the CLI repo.

[nathan-weinberg]

@RobotSail uh, the code is exactly the same:

• https://github.com/instructlab/instructlab/blob/main/.github/workflows/e2e-nvidia-l4-x1.yml
• https://github.com/instructlab/training/blob/main/.github/workflows/e2e-nvidia-l4-x1.yml

how is this specific/different?

[RobotSail]

@nathan-weinberg What I'm saying is that the github action files defining the e2e tests are physically located in this repo. Although the script we're running lives in the instructlab repo, we still define the tasks here separately from the upstream repo.

[RobotSail]

We should keep this issue here as the needs of the training repo may be different from that of the instructlab repo & the SDG repo.