Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support of AutoModel #192

Merged
merged 17 commits into from
Dec 18, 2024
Merged

Support of AutoModel #192

merged 17 commits into from
Dec 18, 2024

Conversation

quic-amitraj
Copy link
Contributor

@quic-amitraj quic-amitraj commented Dec 6, 2024 •
edited
Loading

Adding support of AutoModel

  1. Added support for Embedding models with different architecture.
  2. Upgraded ONNX opset version from 13 to 14.
  3. Added tests to compare and validate outputs across PyTorch, ONNX, and AI 100 runtime.

@quic-amitraj quic-amitraj self-assigned this Dec 6, 2024
@quic-amitraj quic-amitraj marked this pull request as draft December 6, 2024 08:55
@quic-amitraj quic-amitraj added enhancement New feature or request wip Work in progress model-enablement labels Dec 7, 2024
@quic-amitraj quic-amitraj marked this pull request as ready for review December 9, 2024 18:39
@quic-amitraj
Copy link
Contributor Author

Currently, the following models and architectures are validated with this PR:

  1. nomic-ai/nomic-embed-text-v1.5 - NomicBertModel
  2. sentence-transformers/multi-qa-mpnet-base-cos-v1 - MPNetForMaskedLM
  3. BAAI/bge-reranker-v2-m3 - XLMRobertaForSequenceClassification
  4. BAAI/bge-small-en-v1.5 - BertModel

@quic-amitraj quic-amitraj changed the title Adding support of embedding Models Adding support of AutoModel Dec 9, 2024
@quic-amitraj quic-amitraj changed the title Adding support of AutoModel Support of AutoModel Dec 9, 2024
@quic-amitraj quic-amitraj removed the wip Work in progress label Dec 9, 2024
ochougul
ochougul requested changes Dec 10, 2024
View reviewed changes
QEfficient/generation/text_generation_inference.py Outdated Show resolved Hide resolved
QEfficient/generation/text_generation_inference.py Outdated Show resolved Hide resolved
tests/transformers/models/test_causal_lm_models.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Show resolved Hide resolved
quic-amitraj and others added 6 commits December 11, 2024 21:48
@quic-amitraj
Docker-driven tests with latest SDKs (quic#180)
0b80dac 
* Added Docker support to the Jenkins tests

Signed-off-by: amitraj <[email protected]>

* Addressed comments

Signed-off-by: amitraj <[email protected]>

* updated qaic tests time upper limit to 60 minutes

Signed-off-by: Onkar Chougule <[email protected]>

---------
Signed-off-by: amitraj <[email protected]>

* Added support for Embedding moodels
@quic-amitraj
Added support for embedding models
e83b29a 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Lint & Format
be59287 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Added batch_size
1284155 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Docstring added
3f95df7 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Fix-1
74ffc16 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Comments Addressed-1
2fb41ad 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Comments addressed-2
262f45e 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Lint and formatted
ba0258b 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Comments addressed-3
ba66c75 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Fix-2
38a4186 
Signed-off-by: amitraj <[email protected]>
anujgupt-github
anujgupt-github reviewed Dec 16, 2024
View reviewed changes
Copy link
Contributor

@anujgupt-github anujgupt-github left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add updates to the README also in the PR.

QEfficient/generation/text_generation_inference.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
@anujgupt-github
Copy link
Contributor

the C++ execution path also needs to be adapted for embedding models.
@asmigosw

@quic-amitraj
Comments addressed-4
4401fd6 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Minor fix-1
206c81a 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
fix-major
0f1f8bb 
Signed-off-by: amitraj <[email protected]>
ochougul
ochougul requested changes Dec 18, 2024
View reviewed changes
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
QEfficient/transformers/models/modeling_auto.py Outdated Show resolved Hide resolved
tests/transformers/models/test_embedding_models.py Outdated Show resolved Hide resolved
tests/transformers/models/test_embedding_models.py Outdated Show resolved Hide resolved
@quic-amitraj
fix-minor-2
6c9de4b 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
fix-minor-3
88e0fe6 
Signed-off-by: amitraj <[email protected]>
@quic-amitraj
Update ONNX_EXPORT_OPSET to 13
157142a 
Signed-off-by: amitraj <[email protected]>
@ochougul ochougul merged commit 26e472e into quic:main Dec 18, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anujgupt-github anujgupt-github anujgupt-github left review comments

@quic-rishinr quic-rishinr quic-rishinr requested changes

@ochougul ochougul ochougul requested changes

Assignees

@quic-amitraj quic-amitraj

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@quic-amitraj @anujgupt-github @ochougul @quic-rishinr