diff --git a/docs/gptneox_guide.md b/docs/gptneox_guide.md
index dcedbe8ed..45c7f0d13 100644
--- a/docs/gptneox_guide.md
+++ b/docs/gptneox_guide.md
@@ -7,12 +7,14 @@
   - [Introduction](#introduction)
     - [Inference Options](#inference-options)
     - [Supported features](#supported-features)
-  - [Setup](#setup)
+  - [Setup from EleutherAI checkpoint](#setup-from-eleutherai-checkpoint)
     - [Requirements](#requirements)
     - [Download the model](#download-the-model)
     - [Tokenizer](#tokenizer)
+  - [Setup from HuggingFace checkpoint](#setup-from-huggingface-checkpoint)
     - [Run GPT-NeoX](#run-gpt-neox)
 
+
 ## Introduction
 
 This document describes the steps to run the GPT-NeoX model on FasterTransformer.