Convert HF Llama Checkpoints to Neox Checkpoints

[sxthunder]

Hello, I am excited that gpt-neox now support llama model. However, the script in tools/convert_raw_llama_weights_to_neox.py only support origin llama weight. Considering the large number of users currently using Huggingface, would it be possible to provide a script for converting the Huggingface Llama model into Neox?

In my experiments, training speed and memory usage in gpt-neox is much better than other language model framework, even if training by Lora. So I want to use gpt-neox to train if it support the model.

[StellaAthena]

@haileyschoelkopf You were the one who ported LLaMA2 right? Do you think it would be easy to adapt that script to porting LLaMA1?