Inpainting Pipeline Implementation on ai-runner Bounty [$850]

[JJassonn69]

Overview

We’re excited to expand the capabilities of our AI-network by implementing an Inpainting pipeline! This addition will unlock advanced image manipulation features, boosting the functionality and versatility of our AI tools. 🔥

We invite the community to contribute to this exciting initiative by helping integrate the Inpainting pipeline into the AI-worker component of the network. Your contribution will enhance image editing capabilities on the Livepeer network, pushing the boundaries of what's possible with (decentralised) AI-driven media. 🚀

Required Skillset

• Familiarity with FastAPI.
• Strong Python programming skills.
• Familiar with docker and Hugging Face 🤗

Bounty Requirements

1. Implementation: Develop a working /in-paint route and pipeline in the AI-worker repository. This pipeline should be accessible on docker port 8600.
2. Functionality: The pipeline must accept an initial image, image mask and text input as prompt and return modified image as result. Also optional parameters such as negative_prompt, blur_factor, strength, guidance_scale should be tuneable according to the inpaining model used.

Scope Exclusions

• This bounty does NOT cover the complete end-to-end implementation of this pipeline on the go-livepeer side, including payment logic and job routing. These aspects will be addressed by the AI SPE team or in a future bounty.

Implementation Tips

To learn how to create a new pipeline, refer to the Hugging Face pipeline documentation. You can also explore the following pull requests to see how other pipelines were implemented:

• Segment Anything 2 Pipeline Implementation
• Audio to Text Pipeline Implementation

In some cases, you may not be able to integrate the new pipeline directly into the regular AI Runner Docker container due to dependency conflicts or missing packages. If this occurs, you can follow the approach outlined in this example to create a custom container for the pipeline. This approach uses the regular AI runner as the base while keeping the base container lean.

Additionally, keep the following best practices in mind:

• Leverage Existing Work: Explore the Hugging Face documentation on inpainting, such as this guide and this project. These resources can provide useful insights and a solid foundation for your implementation.

• Use Developer Documentation: Be sure to consult our developer documentation for both the worker and runner. These resources can help you speed up your development by offering tips on mocking pipelines and enabling direct debugging.

• Generate the OpenAPI Spec: Run the runner/gen_openapi.py script to create an updated OpenAPI specification for your pipeline.

• Generate Go-Livepeer Bindings: In the main repository folder, run the make command to generate the necessary bindings, ensuring seamless integration with the go-livepeer repository.

How to Apply

1. Express Your Interest: Comment on this issue to indicate your interest and explain why you're the ideal candidate for the task.
2. Wait for Review: Our team will review expressions of interest and select the best candidate.
3. Get Assigned: If selected, we'll assign the GitHub issue to you.
4. Start Working: Dive into your task! If you need assistance or guidance, comment on the issue or join the discussions in the #developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on this GitHub issue when your pull request is ready for review.
7. Receive Your Bounty: We'll arrange the bounty payment once your pull request is approved.
8. Gain Recognition: Your valuable contributions will be showcased in our project's changelog.

Thank you for your interest in contributing to our project! 💛

Warning

Please wait for the issue to be assigned to you before starting work. To prevent duplication of effort, submissions for unassigned issues will not be accepted.

[rickstaa]

LGTm thanks 🙏🏻 .

[JJassonn69]

Posted to livepeer/bounties. Closing it here