Merge multiple LoRA adapters per request (linear, TIES, DARE)

[tgaddair]

Closes #159.

Takes large parts of the implementation from huggingface/peft#1364.

Notably, this approach merges adapters statically during load, rather than during the forward pass. The benefit is that if you make multiple calls to the same merge params, it will be much faster, but potentially slightly slower if the user wants to frequently change merge params each request.

Usage:

"merged_adapters": {
    "ids": ["smangrul/tinyllama_lora_norobots", "smangrul/tinyllama_lora_sql", "smangrul/tinyllama_lora_adcopy"],
    "weights": [2.0, 0.3, 0.7],
    "merge_strategy": "ties",
    "density": 0.2,
    "majority_sign_method": "total"
}

[flozi00]

Great addition
Lorax Moe coming 😂

[prateeky2806]

Hi @tgaddair, I am Prateek the author of TIES-Merging. This is super great that people find TIES and its follow-ups like DARE useful and thanks for integrating this in the library.

I see that you have already added the merging methods to the Docs, however, it would also be great if there is some mention of it on the GitHub Readme so that people know that this feature is added and can read more about it. Let me know what you think about this and if you need help with this.

Thanks for your amazing library and work!
Prateek Yadav

[tgaddair]

Hey @prateeky2806, thanks for your work on TIES-Merging and for your suggestion here! I agree, mentioning this on the README is a good idea. It's something I was hoping to do as a follow-up, as I wasn't sure what the best place to put it would be, but I should be able to put something together in the next day or two.

[tgaddair]

Added #218 to track README updates.

[prateeky2806]

Hi @tgaddair, I think this should be fixed similar to what was done in the HF PEFT library. In their initial commit, they had the wrong ordering of the steps. I would suggest to maybe look at the discussion and fixed here.

Basically, the weight step should be done after obtaining the majority sign.

[tgaddair]

Thanks @prateeky2806! Let me put together a quick PR.

[prateeky2806]

FYI you would also need to change the order of the steps in dare_ties.

[tgaddair]

@prateeky2806 can you take a look at #239 and verify the fix looks correct? Thanks again for pointing this out!

[prateeky2806]

Hi @tgaddair, these changes look good, there is just one last thing. It would be ideal to set the default weight for TIES to all ones because that a setting we experimented with in our paper and kind of works well if people do not want to try out different values.

[prateeky2806]

I have also left a similar comment in the HF PEFT pull request which I think will be fixed in the coming days.

[tgaddair]

Thanks @prateeky2806, I believe setting default weights to 1s is handled here: https://github.com/predibase/lorax/blob/main/server/lorax_server/utils/merges/strategies.py#L107

But let me know if I am misunderstanding something. Happy to change if needed.

[prateeky2806]

Thanks @tgaddair, this looks correct. I missed this when I was going over the code.
Everything looks correct to me now.

[tgaddair]

Thanks for verifying @prateeky2806!