This fine-tunes the GPT-J 6B model on the Alpaca dataset using a Databricks notebook. Please note that while GPT-J 6B is Apache 2.0 licensed, the Alpaca dataset is licensed under Creative Commons NonCommercial (CC BY-NC 4.0).
To use:
- Load all the files into a Databricks repo.
- Start a
12.2 LTS ML (includes Apache Spark 3.3.2, GPU, Scala 2.12)single-node cluster with node type having 8 A100 GPUs (e.g.Standard_ND96asr_v4orp4d.24xlarge). - Open the
train_dollynotebook and run it.