SD3 Medium GPU Demo

Run Stable Diffusion 3 Medium as a Replicate Cog on Fly.io!

Stable Diffusion 3 Medium is Stability AI's most advanced text-to-image open model yet. This demo exposes the model via a simple HTTP server, thanks to Replicate Cog. Cog is an open-source tool that lets you package machine learning models in a standard, production-ready container. When you're up and running, you can generate images using the /predictions endpoint. Images are automatically stored in object-storage on Tigris (you'll need to make sure you add a bucket to the app).

Deploy to Fly.io

Important

Before you deploy, you'll need to get access to the model on Hugging Face by filling out the form in the model repo.

fly apps create --name [APP_NAME]

fly storage create

fly secrets set HUGGING_FACE_HUB_TOKEN=<HUGGING_FACE_TOKEN>

cog push registry.fly.io/[APP_NAME]:latest --use-cuda-base-image false

Now replace image in your fly.toml, then:

fly deploy

Example Request

curl --silent https://[APP_NAME].fly.dev/predictions \
    --request POST \
    --header "Content-Type: application/json" \
    --data @- << 'EOF'
{
    "input": {
        "prompt": "a magical forest",
        "aspect_ratio": "3:2",
        "output_quality": 100
    }
}
EOF

Now view your image at https://fly.storage.tigris.dev/[BUCKET_NAME]/[OUTPUT_IMAGE_NAME]

How can I make the endpoint private?

Having trouble?

Create an issue or ask a question here: https://community.fly.io/