add NPU support for huggingface.py

[jiaqiw09]

what this PR do

issue: #1797
This PR add NPU support for huggingface.py. It just does some fix of existing code to support NPU device.

what part to fix

Currently, the class HFLM just support three different ways to do evaluations:

• using single card just set cuda:0
• using accelerate to do evaluation on multiple cards
• using device_map = 'auto' to do evaluation on multiple cards

how to fix and why

Here are explanation of my code:

- using single card just set cuda:0

Just simply add ["npu"] and ["npu:0"] to device_list as mps. If users want to use in different card, they can export ASCEND_RT_VISIBLE_DEVICES =1 or 2 or 3 and device npu to run task.

- using accelerate to do evaluation on multiple cards

The major change is just replace f"cuda:{accelerator.local_process_index}" with f"{accelerator.device}", it does the same thing, and I think it may help adapt more different devices later if accelerate supports.

- using device_map = 'auto' to do evaluation on multiple cards

For device_map = 'auto', there is something different. If people want to use device_map = 'auto' in NPUs, they can use following code

lm_eval --model hf \
    --tasks lambada_openai,arc_easy \
    --model_args parallelize=True \
    --device npu:0 \
    --batch_size 16

the card info should be set. I do this because of this issue, I met same problem in NPUs. And the problem could be solved by setting specific card. I think it's better just set the device.

[CLAassistant]

All committers have signed the CLA.

[jiaqiw09]

@haileyschoelkopf @lintangsutawika

would you mind having a check?
Best

[haileyschoelkopf]

Hi @jiaqiw09 !

• using accelerate to do evaluation on multiple cards

The major change is just replace f"cuda:{accelerator.local_process_index}" with f"{accelerator.device}", it does the same thing, and I think it may help adapt more different devices later if accelerate supports.

This is a good change. For the rest though:

• could we rename device_counts either back to gpus or to device_count?
• we should be able to handle "npu:{i}". as @LSinev mentions.

Before any merge of this we'd want to be sure we can 1) make certain that this code isn't going to break any existing integrations and 2) try to keep this logic minimally invasive--currently it seems to rely on Accelerate and torch NPU support which I presume is recent. I don't want these to make requirements for installing the library more unwieldy.

Actually, would you be able to point out to me where NPU support lives in torch, or the easiest path to installing such? It's my understanding that as of #1470 this support was not native, in which case I'd be wary of merging code that requires torch.npu if this is still the case.

[jiaqiw09]

@haileyschoelkopf @lintangsutawika

thanks for your suggestion and thanks @statelesshz support in fixing code. I have just test code in NPU and GPU, three methods all work. Would you mind having a look again?

Best

[ji-huazhong]

cc @haileyschoelkopf