Issues running with gpt4-x-alpaca-native

[MarkSchmidty]

This is a 13B full finetune, not a peft, usinf a large GPT-4 dataset on top of a previous full finetune of Alpaca (cleaned) 13B.

It can be found in GPTQ 4bit .pt format here: https://huggingface.co/anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g/tree/main

I ran into a lot of issues trying to get it to work. But I figure @manyoso can probably easily swap out the above for Alpaca peft for a quick test and post the results. I'm curious to see how this one performs. 

(Original gpt4-x-alpaca in 16bit can be found here: https://huggingface.co/chavinlo/gpt4-x-alpaca on the creator's HuggingFace)

[manyoso]

What issues did you find? I can try and run it in the next week of so

[MarkSchmidty]

Nothing major. I tried renaming it to llama7b and launching it as llama7b. But llama needs to be prompted like Alpaca, which currently expects a peft. I'm not quite sure how to do that.