Skip to content

[Feat] Allow symmetric_no_clipping_error for KleidiAI kernels, update Readme and validate Kleidi INT4 quantization path #2570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Jul 26, 2025
Merged

[Feat] Allow symmetric_no_clipping_error for KleidiAI kernels, update Readme and validate Kleidi INT4 quantization path #2570

merged 7 commits into from
Jul 26, 2025

Conversation

gausah-arm
Copy link
Contributor

@gausah-arm gausah-arm commented Jul 18, 2025 •
edited
Loading

This PR validates the support for the INT4 quantization path using KleidiAI by checking the integration to use the new quantize_() API and Int8DynamicActivationIntxWeightConfig. The PR addresses breakages like Symmetric_no_clipping_err caused by API refactors and ensures compatibility with the current layout.

@gausah-arm
[Feat] Restore and validate KleidiAI INT4 quantization path using upd…
25aa861 
…ated quantizer API

- Switched to quantize_() with Int8DynamicActivationIntxWeightConfig
- Validated the move of
  packed_linear_int8_dynamic_activation_intx_weight_layout.py in torchao/dtypes/uintx
- Fixed handling of SYMMETRIC_NO_CLIPPING_ERR mapping type
- Validated INT4 path on a 2-layer nn.Sequential model with torch.int4 weights
- Compared SYMMETRIC vs SYMMETRIC_NO_CLIPPING_ERR across PerAxis and PerGroup granularities
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 18, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2570

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 3 Cancelled Jobs, 9 Pending

As of commit c4c1e50 with merge base 2eb4f97 (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOBS - The following jobs were cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla Meta CLA
Copy link

meta-cla bot commented Jul 18, 2025

Hi @gausah-arm!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

@gausah-arm gausah-arm changed the title [Feat] Restore and validate KleidiAI INT4 quantization path using updated quantizer API [Feat] Allow symmetric_no_clipping_error for KleidAI kernels, update Readme and validate Kleidi INT4 quantization path Jul 21, 2025
@gausah-arm gausah-arm changed the title [Feat] Allow symmetric_no_clipping_error for KleidAI kernels, update Readme and validate Kleidi INT4 quantization path [Feat] Allow symmetric_no_clipping_error for KleidiAI kernels, update Readme and validate Kleidi INT4 quantization path Jul 21, 2025
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 21, 2025
@nikhil-arm
Copy link
Contributor

@digantdesai @jerryzh168 @metascroy can you please review this PR

@digantdesai digantdesai requested a review from metascroy July 23, 2025 16:02
@digantdesai
Copy link
Contributor

@metascroy - can you please take a look?

metascroy
metascroy approved these changes Jul 23, 2025
View reviewed changes
Copy link
Contributor

@metascroy metascroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall changes look great! Let’s see what CI says.

@metascroy
Copy link
Contributor

metascroy commented Jul 23, 2025 •
edited
Loading

@nikhil-arm can you fix the linter error (ruff) and then we can land.

You could also add the new mapping type to the tests here to get CI coverage: https://github.com/pytorch/ao/blob/main/torchao/experimental/tests/test_int8_dynamic_activation_intx_weight.py#L56

@gausah-arm
Copy link
Contributor Author

Overall changes look great! Let’s see what CI says.

Thankyou so much!

@gausah-arm
Copy link
Contributor Author

@nikhil-arm can you fix the linter error (ruff) and then we can land.

You could also add the new mapping type to the tests here to get CI coverage: https://github.com/pytorch/ao/blob/main/torchao/experimental/tests/test_int8_dynamic_activation_intx_weight.py#L56

Thankyou! I have fixed the lint error and done with the changes around test. Please review.

@metascroy
Copy link
Contributor

Running CI. I'll merge if it passes

@metascroy metascroy added the topic: improvement Use this tag if this PR is an improvement (doesn't fit into any of the other categories) label Jul 26, 2025
@metascroy metascroy merged commit ebfe173 into pytorch:main Jul 26, 2025
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@metascroy metascroy metascroy approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: improvement Use this tag if this PR is an improvement (doesn't fit into any of the other categories)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@gausah-arm @nikhil-arm @digantdesai @metascroy