Skip to content

Add support for image-to-video task type for Replicate #1575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

zeke
Copy link
Contributor

@zeke zeke commented Jun 30, 2025

This should unblock support for Replicate-powered inference for https://huggingface.co/MeiGen-AI/MeiGen-MultiTalk

cc @lucataco @apolinario @SBrandeis @Wauplin

@SBrandeis
Copy link
Contributor

Nice, thank you!

Is the end goal to map to https://replicate.com/zsxkib/multitalk/api/api-reference ? if so, the proposed change won't work (it does not match the API of model on Replicate).

Unfortunately there don't seem to be a standard API for image-to-video on Replicate (iiuc every model has a different, specific API), which will require editing this client code everytime a model needs to be supported.
For example, the API for https://replicate.com/kwaivgi/kling-v2.1/api/api-reference and https://replicate.com/zsxkib/multitalk/api/api-reference differ quite significantly.

Until Replicate all those models converge to a common, standard API, I don't think it will be possible for us to properly support the image-to-video task 😞

@SBrandeis
Copy link
Contributor

Note: I opened #1578 to add general support for the task in the Inference client

SBrandeis added a commit that referenced this pull request Jul 1, 2025
Add general support for the `image-to-video` task in
`@huggingface/inference`

cc @zeke (related to #1575 )

---------

Co-authored-by: apolinário <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants