add GenAI_HFLM class to support microservice.

[lkk12014402]

Description

customize a GenAI_HFLM class to request GenAI microservice.

[lkk12014402]

usage

cd GenAIEval/evaluation/lm_evaluation_harness/examples

python main.py \
    --model genai-hf \
    --model_args "base_url=http://localhost:9000,tokenizer=Intel/neural-chat-7b-v3-3" \
    --tasks  "lambada_openai" \
    --batch_size 2

corresponds to this pr to start a llm server

[lkk12014402]

accuracy comparison

use GenAIComps service

start a server with opea-project/GenAIComps#82

python main.py \
    --model genai-hf \
    --model_args "base_url=http://localhost:9000,tokenizer=Intel/neural-chat-7b-v3-3" \
    --tasks  "lambada_openai" \
    --batch_size 2 \
    --limit 100

|    Tasks     |Version|Filter|n-shot|  Metric  |Value |   |Stderr|
|--------------|------:|------|-----:|----------|-----:|---|-----:|
|lambada_openai|      1|none  |     0|perplexity|3.0229|±  |0.4102|
|              |       |none  |     0|acc       |0.7400|±  |0.0441|

original lm-eval

python main.py \
    --model hf \
    --model_args pretrained=Intel/neural-chat-7b-v3-3 \
    --tasks  "lambada_openai" \
    --device cpu \
    --batch_size 2 \
    --limit 100

|    Tasks     |Version|Filter|n-shot|  Metric  |Value |   |Stderr|
|--------------|------:|------|-----:|----------|-----:|---|-----:|
|lambada_openai|      1|none  |     0|perplexity|3.0229|±  |0.4102|
|              |       |none  |     0|acc       |0.7400|±  |0.0441|

note: You can expect results to vary slightly for different input_ids, which has an expect on the inference like position embedding:

• GenAIComps accepts full input_ids
• original lm-eval use input_ids[:-1]