-
Notifications
You must be signed in to change notification settings - Fork 1
OV POT integration for QuestionAnswering task #4
base: main
Are you sure you want to change the base?
Conversation
task: squad | ||
batch_size : 1 | ||
max_length : 384 | ||
calib_size: 50 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that parameter regulates a number of samples in dataset? BTW, should we specify stat_subset_size
as well to specify a number of samples for quantization.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, actually, stat_subset_size can be specified through the config file as well..I will remove the calib_size parameter since its redundant
@@ -0,0 +1,14 @@ | |||
model: | |||
model_ir_path : 'model_ir' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if we can use cache directory to store IR files. Let me check and add update here.
There is an error on CI:
Please workaround it by torch.sqrt = lambda x: torch.pow(x, 0.5) |
runs-on: ubuntu-18.04 | ||
strategy: | ||
matrix: | ||
transformers-version: [4.9.1, 4.11.0, latest] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's test only 4.11.0:
python -m pip install transformers==4.11.0 torch==1.9.1
Hi, @karkadad! Do you mind if I push to this branch to resolve CI issues? |
@dkurt Please go ahead, thanks |
What does this PR do?
This PR includes an implementation to integrate OpenVINO POT APIs for the QuestionAnswering task for 8-bit post-training quantization.
@dkurt Please review.
Before submitting