LoRA injection utils #3
Conversation
RyanJDick
force-pushed
the
ryan/lora-injection
branch
from
42c3ad4
to
330bf2f
Compare
| from invoke_training.lora.lora_block import LoRABlock | ||
|
|
||
|
|
||
| def find_modules( |
There was a problem hiding this comment.
How do you plan on deciding which device to use with torch? In the main app we do a lot of passing around a torch.device object between functions. Could be nice if there's a cleaner way.
There was a problem hiding this comment.
Good call out. I hadn't thought about that yet here. As a start, I think I'll update from_layer(...) to copy the device from the original layer. I can't think of a use case for wanting to override that, but it would be easy enough to add a device parameter to inject_lora_layers(...) in the future if we wanted to.
I'll make this change in PR #2
There was a problem hiding this comment.
could be a fast follow in a new PR to get everything consolidated into main for now
There was a problem hiding this comment.
Addressed in 2225965
Add utils and classes to support injecting LoRA layers into existing networks.
main