[Question] How to use FinGPT open source models from huggingface?

[vineetaparodkar]

I am exploring Financial LLMs and trying out FinGPT but not sure on how to go about to try out some simple queries. Below is sample code can anybody guide on what needs to be done to be able to use the FinGPT model in applications as is without retraining the model.

Below code throws 401 authentication error can anyone guide on how to consume the models.

Error - /root/fingpt-handson/myenv/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Traceback (most recent call last):
  File "/root/fingpt-handson/myenv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/root/fingpt-handson/myenv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/FinGPT/FinGPT-F/resolve/main/tokenizer_config.json

The above exception was the direct cause of the following exception:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# Set devices
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the pre-trained FinGPT-F model & tokenizer
model_name = "FinGPT/FinGPT-F"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name).to(device)
model.eval()

# Input question
question = "What is Microsoft's estimated Q4 FY2024 revenue?"
inputs = tokenizer(question, return_tensors="pt").to(device)

# Generate answer
outputs = model.generate(
    inputs["input_ids"],
    attention_mask=inputs["attention_mask"],
    max_length=100,
    min_length=50,
    early_stopping=True,
    do_sample=False,
)

answer = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(answer)```