Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix winclip openvino inference output without mask #2083

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

junxnone
Copy link

📝 Description

WinClip OpenVINO Inference output is only the classification score.

✨ Changes

Select what type of change your PR is:

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • 🔨 Refactor (non-breaking change which refactors the code base)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

  • 📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
  • 📚 I have made the necessary updates to the documentation (if applicable).
  • 🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

@adrianboguszewski
Copy link
Contributor

@junxnone please check if it doesn't conflict with the order from DFM: #1952

@junxnone
Copy link
Author

  • cannot export the torch model for winclip:
anomalib_winclip/train.py", line 21, in <module>
    export_torch = engine.export(model,ExportType.TORCH)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/anomalib_org/src/anomalib/engine/engine.py", line 937, in export
    exported_model_path = model.to_torch(
                          ^^^^^^^^^^^^^^^
  File "/anomalib_org/src/anomalib/models/components/base/export_mixin.py", line 83, in to_torch
    torch.save(
  File "/miniforge3/envs/tanomalib_org/lib/python3.11/site-packages/torch/serialization.py", line 628, in save
    _save(obj, opened_zipfile, pickle_module, pickle_protocol, _disable_byteorder_record)
  File "/miniforge3/envs/tanomalib_org/lib/python3.11/site-packages/torch/serialization.py", line 840, in _save
    pickler.dump(obj)
AttributeError: Can't pickle local object 'WinClipModel.encode_image.<locals>.get_feature_map.<locals>.hook'

  • my export code(based on the main branch of anomalib)
from anomalib.data import MVTec
from anomalib.models import WinClip
from anomalib.engine import Engine
from anomalib.deploy import ExportType

datamodule = MVTec()
model = WinClip()
engine = Engine()

engine.fit(datamodule=datamodule, model=model)

export_torch = engine.export(model,ExportType.TORCH)

@junxnone
Copy link
Author

junxnone commented May 27, 2024

root cause for this issue is here:

  • only get the first output as output_blob :
    • compiled_model.outputs: [<ConstOutput: names[output] shape[?] type: f32>, <ConstOutput: names[13015] shape[?,240,240] type: f32>]

output_blob = compile_model.output(0)

self.input_blob, self.output_blob, self.model = self.load_model(path)

    11:22:44.293023 line       151         output_blob = compile_model.output(0)
    New var:....... output_blob = <ConstOutput: names[output] shape[?] type: f32>
  • if the shape of the first output is 1, will set the task to CLASSIFICATION

predictions = predictions[self.output_blob]
# Initialize the result variables.
anomaly_map: np.ndarray | None = None
pred_label: LabelName | None = None
pred_mask: float | None = None
# If predictions returns a single value, this means that the task is
# classification, and the value is the classification prediction score.
if len(predictions.shape) == 1:
task = TaskType.CLASSIFICATION
pred_score = predictions

    11:22:45.720814 line       258         predictions = predictions[self.output_blob]
    Modified var:.. predictions = array([0.5035419], dtype=float32)
    11:22:45.721448 line       261         anomaly_map: np.ndarray | None = None
    New var:....... anomaly_map = None
    11:22:45.721725 line       262         pred_label: LabelName | None = None
    New var:....... pred_label = None
    11:22:45.721992 line       263         pred_mask: float | None = None
    New var:....... pred_mask = None
    11:22:45.722259 line       267         if len(predictions.shape) == 1:
    11:22:45.722547 line       268             task = TaskType.CLASSIFICATION
    New var:....... task = <TaskType.CLASSIFICATION: 'classification'>
  • So, OpenVINO Inferencer will output the pred score only

@junxnone
Copy link
Author

Looks like the pred_score for segmentation task is not the correct way to get:

if len(predictions.shape) == 1:
task = TaskType.CLASSIFICATION
pred_score = predictions
else:
task = TaskType.SEGMENTATION
anomaly_map = predictions.squeeze()
pred_score = anomaly_map.reshape(-1).max()

This reverts commit 2a9691fed6065ee4061707eadc4b80df79a51902.
Signed-off-by: junxnone <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task]: Enable WinCLIP export
2 participants