✨ invoke caikit -> peft conversion at load time #161
Merged
+253
−37
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This creates new tempdirs and invokes the caikit to peft conversion utility at prompt adapter load time when a
decoder.pt
format model is detected.How Has This Been Tested?
Loaded up onto a dev pod and tested with the
bloom_sentiment_1
fixture that was added with the conversion utility. Seems to work fine!Need to add a unit test for this specifically, still have to get nox up and running locally. Opening this PR now for feedback.
Merge criteria: