Skip to content

Latest commit

 

History

History
104 lines (76 loc) · 5.73 KB

README_en.md

File metadata and controls

104 lines (76 loc) · 5.73 KB

English | 简体中文

Features | Installation | Quick Start |

PaddleMIX application example is developed based on Paddlevlp, ppdiffusers, and Paddlenlp，which is simple and easy to use and powerful. Aggregating industry high-quality pre trained models and providing out of the box development experience, covering cross modal and multi scenario model library matching, can meet the needs of developers flexible customization .

Features

Out-of-Box Toolset

Multi Modal And Scenario

Out-of-Box Toolset

Appflow provides a rich set of out of the box tools that cover cross modal and multi scenario applications, providing industry level effects and ultimate reasoning performance.

from paddlemix.appflow import Appflow

paddle.seed(1024)
task = Appflow(app="text2image_generation",
               models=["stabilityai/stable-diffusion-v1-5"]
               )
prompt = "a photo of an astronaut riding a horse on mars."
result = task(prompt=prompt)['result']

Multi Modal And Scenario

name models static mode
开放世界检测分割（Openset-Det-Sam） grounded sam ✅
自动标注（AutoLabel） blip2 grounded sam ✅
检测框引导的图像编辑（Det-Guided-Inpainting） chatglm-6b stable-diffusion-2-inpainting grounded sam ✅
文图生成（Text-to-Image Generation） runwayml/stable-diffusion-v1-5 fastdeploy
文本引导的图像放大（Text-Guided Image Upscaling） ldm-super-resolution-4x-openimages ❌
文本引导的图像编辑（Text-Guided Image Inpainting） stable-diffusion-2-inpainting fastdeploy
文本引导的图像变换（Image-to-Image Text-Guided Generation） stable-diffusion-v1-5 fastdeploy
文本条件的视频生成（Text-to-Video Generation） text-to-video-ms-1.7b ❌

More applications under continuous development......

Installation

requirements

pip install -r requirements.txt

For more detailed tutorials on PaddlePaddle and PaddleNLP installation, please refer to Installation。

Source code installation

git clone https://github.com/PaddlePaddle/PaddleMIX
pip install -e .

#appflow requirements
pip install -r paddlemix/appflow/requirements.txt

Quick Start

Taking open world detection segmentation as an example:

Appflow

PaddleMIX provides Appflow without training, and can directly input data to output results:

>>> from paddlemix.appflow import Appflow
>>> from ppdiffusers.utils import load_image

>>> task = Appflow(task="openset_det_sam",
                   models=["GroundingDino/groundingdino-swint-ogc","Sam/SamVitH-1024"],
                   static_mode=False) #如果开启静态图推理，设置为True,默认动态图
>>> url = "https://paddlenlp.bj.bcebos.com/models/community/CompVis/stable-diffusion-v1-4/overture-creations.png"
>>> image_pil = load_image(url)
>>> result = task(image=image_pil,prompt="dog")

Parameter Description

parameter required meaning
--app Yes app name
--models Yes model list,can be a single model or multiple combinations
--static_mode Option static graph inference, default : False
--precision Option when static_mode == True used，default: fp32, option trt_fp32、trt_fp16