instructlab-training initial release take 2
This introduces the instructlab library as a package in the instructlab
package namespace.
To install it:
pip install instructlab-training
And to install it with flash-attn and other CUDA-dependent packages, you can use
pip install instructlab-training[cuda]
Here's how to use it:
from instructlab.training.config import TorchrunArgs, TrainingArgs, run_training
torchrun_args = TorchrunArgs(
nproc_per_node = 1, # 1 GPU
nnodes = 1, # only 1 overall machine in the system
node_rank = 0, # rank of the current machine
rdzv_id = 123, # what ID other nodes will join on
rdzv_endpoint = '0.0.0.0:12345' # address where other nodes will join
)
training_args = TrainingArgs(
# specify training args here
)
run_training(torch_args = torchrun_args, train_args = training_args)