glossar.md

Key Terms

A

Term	Description
Attention Mechanism	A neural network technique for focusing on the most relevant parts of the input sequence, widely used in natural language processing and image recognition.
Autoencoder	A type of neural network used to learn efficient representations by encoding input data and reconstructing it, often used for data compression and denoising.
Activation Function	A mathematical function applied to a neuron's output to introduce non-linearity, such as ReLU, sigmoid, or tanh.

B

Term	Description
Backpropagation	A training method for neural networks that adjusts weights based on the error of each neuron, enabling supervised learning.
Bias	A constant value added to a neuron's output to shift activation and help the model learn from diverse data patterns.
BigGAN	A generative adversarial network that produces high-quality images, known for its large-scale architecture and detailed outputs.

C

Term	Description
Convolutional Neural Network (CNN)	A type of neural network optimized for image data, using convolutional layers to detect patterns.
CycleGAN	A GAN architecture for image translation without paired training examples, often used to transform images between styles.
Conditional GAN (cGAN)	A GAN variant that generates images conditioned on specific input data, such as text labels or images.

D

Term	Description
Discriminator	Part of GANs, this model evaluates the realism of generated data to improve the generator's output.
Diffusion Model	A generative model that gradually adds noise to data and then learns to reverse this process, producing high-quality samples.
Deep Dream	An algorithm that enhances patterns in images by iteratively adjusting neurons in a neural network to maximize output.

E

Term	Description
Encoder	A neural network that maps input data into a latent space, often used in transformers and autoencoders.
Embedding	A representation of data (like words or images) in a high-dimensional vector space for better processing by AI models.
Early Stopping	A technique to prevent overfitting by stopping training once the model's performance on a validation set plateaus or worsens.

F

Term	Description
Feedforward Neural Network	A basic neural network architecture where data flows in one direction from input to output.
Few-Shot Learning	A method that enables models to learn from very few examples by leveraging pre-existing knowledge or features.
Fine-tuning	Adjusting a pre-trained model with additional training on new data to improve performance on specific tasks.

G

Term	Description
Generative Adversarial Network (GAN)	A generative model that pairs a generator with a discriminator to create realistic data, like images or sounds.
Generator	The model in a GAN that learns to produce realistic data samples to fool the discriminator.
GPT (Generative Pre-trained Transformer)	A language model architecture based on transformers, pre-trained on large corpora for various NLP tasks.

H

Term	Description
Hallucination	When a generative model produces content that is not based on the input or is factually incorrect.
Hyperparameters	Adjustable settings in model training, like learning rate or batch size, that significantly impact performance.
Hidden Layer	Intermediate layers in a neural network where data transformations occur before output.

I

Term	Description
Image Synthesis	The process of generating realistic images from random noise or semantic input using generative models.
Inpainting	Filling in missing regions of an image, commonly done with GANs or other image synthesis techniques.
Instance Segmentation	Identifying each distinct object in an image, including pixel-wise differentiation between overlapping instances.

J

Term	Description
Jittering	A data augmentation technique involving random changes in image colors or pixels to improve model robustness.
Joint Attention	Mechanism for focusing on multiple parts of data simultaneously, enhancing model learning in multimodal tasks.

K

Term	Description
k-Nearest Neighbors (k-NN)	A simple algorithm for classification or regression based on the closest training examples in the feature space.
Kernel Trick	A method used in SVMs and other algorithms to make data separable by mapping it into a higher-dimensional space.
Knowledge Distillation	A technique where a smaller model learns from a larger, pre-trained model to retain knowledge efficiently.

L

Term	Description
Latent Space	The abstract, compressed representation of data learned by models, used for generation or manipulation.
LSTM (Long Short-Term Memory)	A type of RNN architecture effective for sequence data, addressing long-term dependency issues.
Loss Function	A metric used during training to assess model prediction error and guide optimization.

M

Term	Description
Multi-Head Attention	A mechanism in transformers that allows focusing on different parts of input data in parallel.
Mode Collapse	A common GAN problem where the generator produces limited diversity, focusing only on a few data modes.
Masked Language Modeling	A pre-training objective where certain tokens are masked and predicted to help models understand context.

N

Term	Description
Neural Network	A computational model with layers of nodes (neurons) inspired by biological neural networks.
Noise Injection	Adding randomness to inputs during training to make models more robust and prevent overfitting.
Normalization	Scaling input data to improve model convergence and reduce training time.

O

Term	Description
Overfitting	When a model learns the training data too well, losing generalization to new data.
Object Detection	Identifying and classifying multiple objects in an image, often using bounding boxes.
Optimization	The process of adjusting model parameters to minimize the loss function during training.

P

Term	Description
Prompt	Input text or instruction guiding the output of a generative model like GPT.
Pre-training	Training a model on large datasets before fine-tuning it for specific tasks, common in language and vision models.
PixelCNN	A generative model that generates images pixel-by-pixel, often used for image synthesis.

Q

Term	Description
Quantization	Reducing the number of bits used in a model's weights to make it faster and more efficient.
Q-Learning	A reinforcement learning technique where agents learn action-value pairs to maximize rewards.
Query	The input in an attention mechanism that is used to find the relevance of other inputs (keys) for prediction.

R

Term	Description
Recurrent Neural Network (RNN)	A type of neural network suited for sequential data, such as text or time series, due to its internal memory.
Reinforcement Learning (RL)	A type of machine learning where agents learn actions in an environment to maximize cumulative rewards.
ResNet	A deep convolutional network architecture with skip connections, allowing very deep networks without performance degradation.

S

Term	Description
Segmentation	The process of dividing an image into different parts or segments to identify and analyze specific objects or regions.
Self-Supervised Learning	A method where models generate their own training labels, enabling learning from unlabeled data.
StyleGAN	A GAN variant used for high-quality, style-controllable image generation, popular in art and media.

T

Term	Description
Transformer	A neural network architecture using self-attention, key in NLP and generative models.
Transfer Learning	Adapting a pre-trained model for a related task, reducing training time and data requirements.
Tokenization	Breaking down input text into smaller units (tokens) that models can process.

U

Term	Description
Unsupervised Learning	A type of learning where the model identifies patterns without labeled data, often for clustering or anomaly detection.
U-Net	A CNN architecture primarily used in biomedical image segmentation, with a U-shaped structure for detailed outputs.
Upsampling	Increasing the resolution of data, often used in image processing for better visual clarity.

V

Term	Description
Variational Autoencoder (VAE)	A type of autoencoder with added randomness, often used for generative tasks involving image and text synthesis.
Vector Quantized (VQ-VAE)	A generative model combining VAEs and quantization to represent data as discrete vectors, useful in image synthesis.
Vision Transformer (ViT)	A transformer-based model applied to vision tasks, often outperforming CNNs in specific contexts.

W

Term	Description
Wasserstein GAN (WGAN)	A GAN variant that improves training stability by using the Wasserstein distance as a loss function.
Word2Vec	A technique for natural language processing that represents words as vectors in a continuous vector space.
Weight Sharing	A method in neural networks where the same weights are used across different parts of the model, often seen in CNNs.

X

Term	Description
Xavier Initialization	A method for initializing neural network weights to avoid gradient vanishing or explosion, especially in deep networks.
XML-RPC	A remote procedure call (RPC) protocol using XML to encode calls, sometimes used for integrating AI systems with external software.
XOR Problem	A classic problem in neural networks involving non-linear classification, motivating the need for multi-layer networks.

Y

Term	Description
YOLO (You Only Look Once)	A real-time object detection algorithm that divides images into grids for fast and accurate predictions.
YAML	A human-readable data serialization format often used in configuring machine learning experiments.
Yield Curve	In financial models, an important time-series input for forecasting, sometimes predicted using generative models.

Z

Term	Description
Zero-shot Learning	A model's ability to recognize and predict classes or tasks it has not been explicitly trained on.
Zero-padding	Adding zeros around image data to maintain dimensional consistency in convolutional neural networks.
Z-score Normalization	A normalization technique for data, scaling values by the mean and standard deviation for improved model performance.