implement torch.linalg.pinv xpu backend

[yao-matrix]

🚀 The feature, motivation and pitch

torch.linalg.pinv is used by PEFT LORA-FA finetuning. Now, it will show below log on xpu:

/usr/local/lib/python3.11/dist-packages/peft/optimizers/lorafa.py:149: UserWarning: Aten Op fallback from XPU to CPU happends. This may have performance implications. If need debug the fallback ops please set environment variable PYTORCH_DEBUG_XPU_FALLBACK=1 (Triggered internally at /pytorch/build/xpu/ATen/RegisterXPU_0.cpp:54525.)
AA_T_inv = torch.linalg.pinv(AA_T + delta * torch.eye(A.shape[0]).to(A.device))

Pls plan xpu backend for this op

Alternatives

No response

Additional context

No response

[CuiYifeng]

Under evaluation.

[CuiYifeng]

@yao-matrix Could you provide the proportion of torch.linalg.pinv in PEFT LORA-FA finetuning E2E time to evaluate priority?