Merge branch 'memory_profiling' of github.com:EleutherAI/gpt-neox into memory_profiling

Author: jahatef

configs/neox_arguments.md

@@ -111,7 +111,7 @@ Logging Arguments
 
 - **git_hash**: str
 
-    Default = 6a8a829
+    Default = f63593b
 
     current git hash of repository
 
@@ -199,6 +199,22 @@ Logging Arguments
 
 
 
+- **memory_profiling**: bool
+
+    Default = False
+
+    Whether to take a memory snapshot of the model. Useful for debugging memory issues.
+
+
+
+- **memory_profiling_path**: str
+
+    Default = None
+
+    Path to save memory snapshot to.
+
+
+
 ## NeoXArgsModel
 
 Model Arguments