[intel-npu] Support new internal cached_model_buffer config for memory mapped cached blobs
#27822
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
category: inference
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
from
ed220a4 to
94e33c4
Compare
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
3 times, most recently
from
539c459 to
8cfb954
Compare
github-actions
bot
removed
category: inference
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
2 times, most recently
from
1f163fa to
5a29e3c
Compare
MirceaDan99
changed the title
import_model API for memory mapped cached blobscached_model_buffer config for memory mapped cached blobs
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
from
5a29e3c to
60a36c4
Compare
pereanub
reviewed
Dec 11, 2024
src/plugins/intel_npu/src/compiler_adapter/include/ze_graph_ext_wrappers.hpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_npu/src/common/include/intel_npu/common/igraph.hpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_npu/src/common/include/intel_npu/common/igraph.hpp
Outdated
Show resolved
Hide resolved
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
6 times, most recently
from
9859b15 to
99b823b
Compare
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
from
99b823b to
122788e
Compare
…or `ov::AlignedBuffer`
…ffer` (no release for mmap shared object)
…unusedStream` when `model_buffer` is given
…>` and `std::shared_ptr<ov::AlignedBuffer>` blob types
…pped buffers
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
from
0ffe5d6 to
d6beace
Compare
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
from
d6beace to
1801088
Compare
`get_blob_container` method
MirceaDan99
force-pushed
the
POC/add_caching_with_mmap_property
branch
from
bcda564 to
02683cf
Compare
PatrikStepan
approved these changes
Jan 22, 2025
…corresponding `Config` implementation
pereanub
approved these changes
Jan 23, 2025
alexandruenache1111
approved these changes
Jan 23, 2025
aobolensk
pushed a commit
to aobolensk/openvino
that referenced
this pull request
Jan 28, 2025
…ory mapped cached blobs (openvinotoolkit#27822) ### Details: - *Based on <s>new `import_model` API from PR openvinotoolkit#27644</s> new plugins property to pass mmap buffer from PR openvinotoolkit#27981* - *Added `BlobContainer` class for `IGraph` objects that may derive with `BlobContainerAlignedBuffer` for the new `import_model` API and `BlobContainerVector` for the old one* - *Refactored `getGraphHandle` function to allow passing `const uint8_t` and `size_t` params instead of `std::vector<uint8_t>* ### Tickets: - *157192* --------- Signed-off-by: Alexandru Enache <[email protected]> Co-authored-by: Alexandru Enache <[email protected]> Co-authored-by: Oleg Pipikin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Details:
newnew plugins property to pass mmap buffer from PR New plugins property to pass mmap buffer #27981import_modelAPI from PR New caching api #27644BlobContainerclass forIGraphobjects that may derive withBlobContainerAlignedBufferfor the newimport_modelAPI andBlobContainerVectorfor the old onegetGraphHandlefunction to allow passingconst uint8_tandsize_tparams instead of `std::vector<uint8_t>Tickets: