llama_model_load: invalid model file './models/ggml-stable-vicuna-13B.q4_2.bin' (bad f16 value 5)

[tianlichunhong]

Below is my code for python:
from pygpt4all import GPT4All

model = GPT4All(r'./models/ggml-stable-vicuna-13B.q4_2.bin')
while True:
try:
prompt = input("You: ", flush=True)
if prompt == '':
continue
print("AI:", end='')
for token in model.generate(prompt):
print(f"{token}", end='', flush=True)
print()
except KeyboardInterrupt:
break

It shows the result is below: bad f16 value 5
llama_model_load: loading model from './models/ggml-stable-vicuna-13B.q4_2.bin' - please wait ...
llama_model_load: n_vocab = 32001
llama_model_load: n_ctx = 512
llama_model_load: n_embd = 5120
llama_model_load: n_mult = 256
llama_model_load: n_head = 40
llama_model_load: n_layer = 40
llama_model_load: n_rot = 128
llama_model_load: f16 = 5
llama_model_load: n_ff = 13824
llama_model_load: n_parts = 2
llama_model_load: type = 2
llama_model_load: invalid model file './models/ggml-stable-vicuna-13B.q4_2.bin' (bad f16 value 5)
llama_init_from_file: failed to load model

file ggml-stable-vicuna-13B.q4_2.bin is download from :https://gpt4all.io/models/ggml-stable-vicuna-13B.q4_2.bin
Can someone help me? Thank you very much!

[rok-GKC]

Same issue here with the vicuna-7b

[tianlichunhong]

It looks like pygpt4all does not support q4 models. But in windows version it work fine.