fix merge conflicts

CONTRIBUTING.md → .github/CONTRIBUTING.md

@@ -129,9 +129,9 @@ When you're ready to contribute code to address an open issue, please follow the
         isort .
         black .
 
-    Our CI also uses [`flake8`](https://github.com/allenai/LLM/tree/main/tests) to lint the code base and [`mypy`](http://mypy-lang.org/) for type-checking. You should run both of these next with
+    Our CI also uses [`ruff`](https://docs.astral.sh/ruff/) to lint the code base and [`mypy`](http://mypy-lang.org/) for type-checking. You should run both of these next with
 
-        flake8 .
+        ruff check .
 
     and
 

.github/actions/setup-venv/action.yml

@@ -19,7 +19,7 @@ runs:
     - shell: bash
       run: |
         # Install prerequisites.
-        pip install --upgrade pip setuptools wheel virtualenv
+        pip install --upgrade pip setuptools build wheel virtualenv
 
     - shell: bash
       run: |

.github/workflows/main.yml

@@ -32,7 +32,7 @@ jobs:
         task:
           - name: Lint
             run: |
-              flake8 .
+              ruff check .
 
         include:
           - python: '3.10'
@@ -49,8 +49,7 @@ jobs:
             task:
               name: Build
               run: |
-                python setup.py check
-                python setup.py bdist_wheel sdist
+                python -m build
 
           - python: '3.10'
             task:
@@ -175,7 +174,7 @@ jobs:
 
       - name: Install requirements
         run: |
-          pip install --upgrade pip setuptools wheel
+          pip install --upgrade pip setuptools wheel build
           pip install -r dev-requirements.txt
 
       - name: Prepare environment

Makefile

@@ -11,7 +11,7 @@ TEST_IMAGE =  $(shell beaker workspace images $(BEAKER_WORKSPACE) --format=json
 run-checks :
 	isort --check .
 	black --check .
-	flake8 .
+	ruff check .
 	mypy .
 	CUDA_VISIBLE_DEVICES='' pytest -v --color=yes tests/
 

README.md

@@ -5,7 +5,7 @@
 After cloning this repository, first install the latest [PyTorch](https://pytorch.org) according the official instructions relevant to your environment. Then install the remaining dependencies and code base by running:
 
 ```
-pip install -e .[dev]
+pip install -e .
 ```
 
 ## Running LM pre-training jobs
@@ -31,25 +31,22 @@ torchrun --nproc-per-node=8 scripts/train.py configs/c4-tiny.yaml \
 
 #### Running on Cirrascale via [beaker-gantry](https://github.com/allenai/beaker-gantry)
 
-Check the script at [`scripts/olmo-small-ablation-on-gantry.sh`](scripts/beaker/olmo-small-ablation-on-gantry.sh) for an example on how to run a training job on Cirrascale.
-
-After installing `beaker-gantry`, you can launch a training job like this:
+Check the script at [`scripts/beaker/olmo-small-ablation-on-gantry.sh`](scripts/beaker/olmo-small-ablation-on-gantry.sh) for an example on how to run a training job on Cirrascale. Using that script, you can launch a training job like this:
 
 ```bash
 CONFIG_PATH=configs/choose_a_config.yml \
 LOAD_PATH=/optional/path/to/checkpoint/ \
   bash scripts/olmo-small-ablation-on-gantry.sh
 ```
 
-if `CONFIG_PATH` is not specified, the default config is `configs/olmo-small-ablation.yaml`;
-if `LOAD_PATH` is not specified, the training will start from scratch.
+If `CONFIG_PATH` is not specified, the default config is `configs/olmo-small-ablation.yaml`. If `LOAD_PATH` is not specified, the training will start from scratch.
 
 #### Running on LUMI via Slurm
 
 First read our [LUMI](docs/LUMI.md) documentation, but submitting a new job essentially just boils down to running this:
 
 ```bash
-sbatch scripts/c4-tiny-on-lumi.sh
+sbatch scripts/lumi/c4-small-on-lumi.sh
 ```
 
 ### Restarting a training job from a checkpoint
@@ -62,7 +59,7 @@ There are also symlinks for the latest checkpoints in the form of `latest` and `
 Sharded checkpoints are the default type of checkpoint that's saved during training since these are the fastest, but you can also save unsharded checkpoints by setting `--save_interval_unsharded [INT]`.
 
 If you plan to restart a training run using a *different* world size, you can only restart from an *unsharded* checkpoint.
-However, you can convert a sharded checkpoint into an unsharded checkpoint by launching the script [scripts/unshard_checkpoint.sh](./scripts/unshard_checkpoint.sh) in the same way you launched the training script. Note that this needs to be launched with the exact same world size as when the *sharded* checkpoint was saved.
+However, you can convert a sharded checkpoint into an unsharded checkpoint by launching the script [scripts/unshard.sh](./scripts/unshard.sh) in the same way you launched the training script. Note that this needs to be launched with the exact same world size as when the *sharded* checkpoint was saved.
 
 ## Finding official runs and checkpoints