This repository contains resources and code for utilizing Latent CLIP for zero-shot prediction and reward-based noise optimization.
To set up the environment, use the provided environment.yml
file:
conda env create -f environment.yml
conda activate latentclipenv
The starting point for understanding and utilizing Latent CLIP is the Jupyter Notebook:
minimal_usage_latent_clip.ipynb
This notebook demonstrates:
- Zero-shot prediction using Latent CLIP.
- Reward-based noise optimization using Latent CLIP-based rewards .
The file workflow_sdxl_turbo.json
is a workflow designed for use with ComfyUI.
Steps to use the workflow:
-
Clone ComfyUI from GitHub:
git clone https://github.com/comfyanonymous/ComfyUI.git cd ComfyUI
-
Run ComfyUI:
python main.py
-
Load the Workflow:
- In the ComfyUI interface, press
Ctrl + O
(or click "Load"). - Select the file
workflow_sdxl_turbo.json
from theassets/
folder.
- In the ComfyUI interface, press
-
For more information on ComfyUI, visit:
➡️ ComfyUI GitHub Repository
supplementary/
│
├── assets/ # Additional resources
│ └── workflow_sdxl_turbo.json # ComfyUI workflow file (https://github.com/comfyanonymous/ComfyUI)
│
├── Latent_ReNO/ # Implementation for reward-based noise optimization
├── environment.yml # Conda environment setup file
├── helper.py # Utility functions for supporting the notebook
└── minimal_usage_latent_clip.ipynb # Main notebook for starting with Latent CLIP