Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft PR for Adding support for Google T5 11b model to ROCm vLLM #318

Closed
wants to merge 15 commits into from
Closed

Draft PR for Adding support for Google T5 11b model to ROCm vLLM #318

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
c0e9b46
Update registry.py
bmullick-amd Dec 2, 2024
d9abfb9
Update preprocess.py
bmullick-amd Dec 2, 2024
e78a495
added t5 model script
bmullick-amd Dec 2, 2024
6cc440b
added source script link
bmullick-amd Dec 18, 2024
124c56b
fix github actions
bmullick-amd Dec 18, 2024
e8b3a9d
fix github actions
bmullick-amd Dec 18, 2024
4f74d72
fixed github actions
bmullick-amd Dec 18, 2024
36a1604
fix github actions
bmullick-amd Dec 18, 2024
a9f54f4
fixed github action ruff fails
bmullick-amd Dec 18, 2024
8b78df7
fixed ruff fails
bmullick-amd Dec 18, 2024
b554266
fix import errors
bmullick-amd Dec 18, 2024
2a8bb2e
fix isort error
bmullick-amd Dec 18, 2024
ea736ed
fixed ruff fails
bmullick-amd Dec 18, 2024
a9d0d8d
yapf format
bmullick-amd Dec 18, 2024
fbaa5fe
fix isort check
bmullick-amd Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion vllm/inputs/preprocess.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ def _prepare_decoder_input_ids_for_generation(
if decoder_input_ids is None:
# no decoder prompt input ->
# use decoder_start_token_id as decoder_input_ids
decoder_input_ids = self._get_default_enc_dec_decoder_prompt()
# decoder_input_ids = self._get_default_enc_dec_decoder_prompt()
decoder_input_ids = [decoder_start_token_id]

if force_bos and (len(decoder_input_ids) == 0
or decoder_input_ids[0] != decoder_start_token_id):
Expand Down
5 changes: 4 additions & 1 deletion vllm/model_executor/models/registry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,9 @@
"BartModel": ("bart", "BartForConditionalGeneration"),
"BartForConditionalGeneration": ("bart", "BartForConditionalGeneration"),
"Florence2ForConditionalGeneration": ("florence2", "Florence2ForConditionalGeneration"), # noqa: E501
"T5Model": ("t5", "T5ForConditionalGeneration"),
"T5ForConditionalGeneration": ("t5", "T5ForConditionalGeneration"),
"T5WithLMHeadModel": ("t5", "T5ForConditionalGeneration")
}

_EMBEDDING_MODELS = {
Expand Down Expand Up @@ -484,4 +487,4 @@ def _run() -> None:


if __name__ == "__main__":
_run()
_run()
Loading
Loading