PolynomialHeadAvatars

CSE 527 Class Project: Polynomial Expression fusion for Instant Volumetric Head Avatars [Spring 2024, Stony Brook University] Submitted by Snehal Singh Tomar

Report:

tomar_snehal_cse527projreport.pdf

Setup:

1. Clone this repository
2. run: conda create -n polyhead python = 3.9
3. run: conda activate polyhead
4. run: pip install -r requirements.txt
5. run: conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
6. run: cd PolynomialHeadAvatars/
7. run: ./install.sh
8. Download 'workspace/' from this link
9. Place 'workspace/' such that its patch is 'PolynomialHeadAvatars/workspace/'
10. Download 'obama.zip' and 'justin.zip' from this link, extract them, and place them such that the following directory structure exists: --PolynomialHeadAvatars/
    |
    |-data/
    | |--obama/
    | |--justin/
    |
11. Set PWD to 'PolynomialHeadAvatars/'

Testing (to reproduce metrics in report)

1. run: python <main_script> <dataset_path> --workspace workspace/<your_experiment_name_of_choice> -O --tcnn --num_rays 2048 --test --ckpt workspace/<checkpoint_path>

2. The specifcs are as below:
   a. dataset path: data/obama or data/justin
   b. main_script:
   i. for vanilla-insta: main_insta.py
   ii. for PNN 1 + PNN 2: main_insta_pnn1_pnn2.py
   iii. for PNN 1 (trimmed) + PNN 2: main_insta_pnn1_2_trimmed.py
   c. checkpoint path:
   i. vanilla-insta:
   -> obama: vanilla-2048-1/checkpoints/ngp.pth
   -> justin: vanilla_justin/checkpoints/ngp.pth
   ii. PNN 1 + PNN 2:
   -> obama: pnn1_2/checkpoints/ngp.pth
   -> justin: pnn1_2_justin/checkpoints/ngp.pth
   iii. PNN 1 (trimmed) + PNN2 (Ours):
   -> obama: pnn1_2_less_params/checkpoints/ngp.pth
   -> justin: pnn1_2_less_params_justin/checkpoints/ngp.pth

Training:

run: python <main_script_per_your_choice> <dataset_path> --workspace workspace/<your_experiment_name_of_choice> -O --tcnn --num_rays 2048

Acknowledgment:

This code is based on INSTA-pytorch.