08-image-models

Generative Images

Autoencoders

• 🎥 What is an Autoencoder? | Two Minute Papers #86
• 📋 Building Autoencoders in Keras

Generative Adversarial Networks (GANs)

Educational Resources

• 🎥 StyleGAN: None of These Faces Are Real!, 🎥 StyleGAN2: Near-Perfect Human Face Synthesis...and More from Two Minute Papers
• 📊 Overview of GAN Structure

Artistic Works

• 🌿 Leaves of Manifold by Helena Sarin
• 🐞 confusing coleopterists / 🤔🐞 breeding bugs in the latent space

Research Papers and Techniques

• 🌱 Progressive Growing of GANs for Improved Quality, Stability, and Variation
• 🎨 Image Style Transfer Using Convolutional Neural Networks
• 🔄 Image-to-Image Translation with Conditional Adversarial Networks
• 🔄 Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• 🎭 A Style-Based Generator Architecture for Generative Adversarial Networks

Diffusion Models

Readings

• 📰 Generative AI Has a Visual Plagiarism Problem
• 📚 LAION-5B: A New Era of Open Large-Scale Multi-Modal Datasets - An overview of the LAION-5B dataset by Romain Beaumont (2022).
• 📊 Models all the Way Down - A visual investigation into the construction of the LAION-5B dataset.

Educational Resources

• 📺 Diffusion models explained in 4-difficulty levels
• 📘 Introduction to Diffusion Models for Machine Learning
• 💻 Progressive Noise Reduction in p5.js example - Demonstration of gradually reducing noise to reveal a target image, simulating how diffusion models denoise step-by-step.

Research Papers

• 📜 Deep Unsupervised Learning using Nonequilibrium Thermodynamics (2015)
• 📜 Denoising Diffusion Probabilistic Models (2020)
• 📜 Improved Denoising Diffusion Probabilistic Models (2021)
• 📜 Diffusion Models Beat GANs on Image Synthesis (2021)

Tools and Interfaces

• 🌌 Text-to-Image Collection on Replicate
• 🖥️ Stable Diffusion WebUI by AUTOMATIC1111 - A web-based interface for running Stable Diffusion models locally.
• 🔧 ComfyUI GitHub Repository - A modular and node-based UI for Stable Diffusion that allows users to experiment with model pipelines.

Code Examples and Implementations

Diffusion Models on Replicate

Examples are shared via email due to usage restrictions at ITP.

Generating images with sd-webui

• 💻 in class sd-webui example with ITP GPU machine (remember, this only works outside the p5.js web editor and on the NYU network or VPN!)

Transformers.js

• 💻 Janus-1.3B-onnx (this is using an unreleased alpha version of transformers.js, proceed with caution!)

Assignment

• Create a final project proposal. See final project guidelines. Check your assigned proposal date and add your proposal to the course wiki.