Analysis and Design of Deep Neural Networks - Homework Submission

Analyse and Design of Deep Neural Network, Dr. Kalhor and Dr. Nadjar Araabi, University of Tehran

About

This repository contains homework submissions for the course "Analysis and Design of Deep Neural Networks," instructed by Dr. Ahmad Kalhor and Dr. Babak Nadjar Araabi.

Course Overview

The course explores the details of designing and analyzing deep neural networks, focusing on three main ideas:

• Separation Index (SI)
• Smoothness Index (SMI)
• Similiraty Learning

Contents

• Homework 1: Data Evaluation - Subset Selection and Feature Selection

  • Techniques for evaluating and selecting subsets of data
    • Separation Index
    • Smoothness Index
  • Feature selection methods to improve model accuracy and efficiency

• Homework 2: Model and Layer Evaluation

  • Evaluation of models and their layers
    • Testing model performance
    • Analyzing layer outputs
    • Understanding layer contributions to final predictions

• Homework Extra: Feature Representation

  • Additional assignment focusing on feature representation techniques
    • Examples of enhancing model learning and prediction accuracy through effective feature representation

• Homework 3: Layer-wise Learning - Image Segmentation Network

  • Implementation and training of segmentation models
  • Train a model with Layer-wise Learning

• Homework 4: Model Compression

  • Exploration of model compression techniques
    • Reducing deep learning model size without significant performance loss
    • Crucial for deployment on devices with limited computational resources

• Homework 5: Metric Learning

  • Coverage of metric learning techniques
    • Triplet loss
    • Contrastive loss
    • Fisher Discriminant Contrastive Loss (FDT)
    • Fisher Discriminant triplet Loss (FCT)
    • Learning from relative comparisons between data points

Included in this repository:

• Code implementations for all homework assignments.
• Detailed reports for each homework assignment.
• Descriptive files providing comprehensive instructions and guidelines for each homework assignment.

Key Features

• Implementation of Separation Index and Smoothness Index found at data_complexity_measures GitHub repository.

Environment and Dependencies

• Python Version: 3.10
• PyTorch Version: 2.1.1+cuda 11.8

Setting up Conda Environment

• Create the Conda environment using the provided YML file (env_addnn.yml):

conda env create -f env_addnn.yml

• Activate the created Conda environment:

conda activate torch2

Acknowledgments

Special thanks to Dr. Ahmad Kalhor and Dr. Babak Nadjar Araabi for their invaluable guidance and instruction throughout the course.

---