Skip to content
This repository has been archived by the owner on Oct 2, 2024. It is now read-only.

handle multiple model artifacts that are associated with a model #9

Closed
Tracked by #7
rbavery opened this issue Feb 14, 2024 · 2 comments
Closed
Tracked by #7

handle multiple model artifacts that are associated with a model #9

rbavery opened this issue Feb 14, 2024 · 2 comments

Comments

@rbavery
Copy link

rbavery commented Feb 14, 2024

As I'm working with AOTIndutor and Torchscript for exporting models, I'm realizing that model publishers will sometimes want to reference runtime details for multiple model artifacts, without duplicating all model extension info.

AOTInductor (.pt2) exports a model with hardware specific optimizations, so it will be tied to a particular accelerator (cpu, gpu, tpu, etc.)

Torchscript tracing (.pt) is hardware agnostic. the loaded model and model inputs just need to be moved to the correct hardware before inference. The optimizations are not hardware specific so accelerator utilization is lower than models compiled with AOTInductor.

Model publishers might want to provide any of combinations of a hardware agnostic model artifact, multiple optimized models, or the original weights.

I think we should probably accept an array of Runtime Objects instead of a single Runtime Object.

@rbavery
Copy link
Author

rbavery commented Feb 15, 2024

done in #2 and updated the hackmd

@fmigneault
Copy link
Collaborator

Various model artifacts should be provided by distinct Assets with mlm:model role.
Each Asset can also provide mlm:artifact_type to be more explicit about the specific artifact content.
Other fields such as mlm:framework can also be applied on individual Assets to allow providing multiple equivalent definitions by various implementations.

Fixed by #2

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants