fix: update documentation and a stylegan3 script

README.md

@@ -51,6 +51,24 @@ Run the following command to lint all python scripts:
 
 * `flake8`
 
+## Model Experiments
+
+We performed experiments with a number of existing models listed below to understand how useful they are in helping
+with generating new Bliss symbols etc.
+
+### StyleGAN3
+
+Conclusion: not useful
+
+Refer to the [documentation](./docs/TrainStyleGAN3MOdel.md) about how to train this model, training results and
+the conclusion about how useful it is.
+
+### Texture Inversion
+
+Concolusion: not useful 
+
+Refer to the [documentation](./notebooks/README.md) for details.
+
 ## Utility Functions
 
 All utility functions are in `utils` directory.

docs/StyleGANModelTraining.md → docs/PrepareImageSetForStyleGAN.md

@@ -1,23 +1,24 @@
-# Train StyleGAN model
+# Prepare Image Set For StyleGAN models
 
 [StyleGAN model](https://machinelearningmastery.com/introduction-to-style-generative-adversarial-network-stylegan/) 
-offers control over the style of the generated image. Bliss has 1217 single characters that are used to compose other
-Bliss words. It is interesting to train a StyleGAN model with these single characters to find out if it is useful to
-give us some new bliss shapes, and to give the whole team a feel for what these systems can and cannot do.
+offers control over the style of the generated image. Bliss has 1217 single characters that are used to compose
+Bliss words. It is interesting to train StyleGAN models with these single characters to find out if they are useful to
+give us some new bliss shapes for brainstorming. It will also provide the team a feel for what these systems can and
+cannot do.
 
-## Prepare the image set
-
-Goal: The image set contains all Bliss single characters. All images need to be transformed to grayscale. They should
-also be cropped or padded out to make a square of 256 x 256 with the baseline always in the same position, and the
-symbol centered horizontally.
+The image set should satisfy these conditions:
+1. It contains all Bliss single characters. 
+2. Images need to be transformed to grayscale. 
+3. Images should be cropped or padded out to make a square of 256 x 256 with the baseline always in the same position, 
+and the symbol centered horizontally.
 
 Note: The Bliss single character with BCI ID 25600 is missing from the final image set. According to the information
 from [Blissary](https://blissary.com/blissfiles/), this character is missing on purpose because it has been decided as
 part of the work with encoding Blissymbolics into Unicode. This particular character will not be part of Unicode. It's
 because it doesn't stand for a concept, it's just a description of a graphical shape used in the Bliss character for
 squirrel.
 
-### Steps
+## Steps
 
 Step 1. At [the Bliss File page](https://blissary.com/blissfiles/), download the png package with height 344px,
 transparent background, 384dpi resolution and the naming of BCI ID.
@@ -91,56 +92,3 @@ The list of images with the second max height is:  []
 
 The verification shows the resizing is correct.
 ```
-
-## Train the styleGAN3 model
-
-[The styleGAN3 model](https://github.com/NVlabs/stylegan3) is trained on [the Cedar platform](https://docs.alliancecan.ca/wiki/Cedar). This model supports [three configs](https://github.com/NVlabs/stylegan3/blob/main/docs/configs.md): 
-StyleGAN3-T (translation equiv.), StyleGAN3-R (translation and rotation equiv.), or StyleGAN2.
-
-### Start the training job
-
-Step 1: Use [`rsync`](https://linuxhandbook.com/transfer-files-ssh/) or other commands to transfer transformed Bliss
-images to Cedar
-
-Step 2. Login to the Cedar and fetch stylegan3 source code
-```
-mkdir stylegan3
-cd stylegan3
-git clone https://github.com/NVlabs/stylegan3
-```
-
-Step 3. Creating a zip archive of Bliss images will lead to a better performance
-```
-cd stylegan3
-python dataset_tool.py --source=../bliss_single_chars_final --dest=../datasets/bliss-256x256.zip
-```
-
-Step 4: Submit Job
-
-* Copy [requirements.txt](../jobs/stylegan3/requirements.txt) to stylegan3 source code root directory.
-
-* Copy [job_stylegan3.sh](../jobs/stylegan3/job_stylegan3.sh) to the `scratch/' directory in your home directory
-
-* Submit the job
-
-```
-cd ~/scratch
-sbatch job_stylegan3.sh
-```
-
-Use `sq` to check the status of the job. Use `scancel` to cancel a running job.
-
-### The training result
-
-The Bliss images were first trained using `stylegan3-r` config (translation and rotation equiv.). This job had to be
-cancelled after running 2.5 days because the cluster the job was running on is a shared resource that was waited by
-another team. Before the cancellation, the job had generated some training result that can be found at
-[this repository](https://github.com/cindyli/bliss-data/tree/main/styleGAN/styleGAN-training-results/stylegan3-r).
-
-`reals.png` is a collection of real Bliss symobles
-
-`fakes*.png` are random image grids exported from the training loop at regular intervals.
-
-`training_options.json` contains training options used for this round of training.
-
-`metric-fid50k_full.jsonl` logs the result and records` FID evaluated by the training loop for every export.

docs/TrainStyleGAN3Model.md

@@ -0,0 +1,70 @@
+# Train StyleGAN model
+
+[The styleGAN3 model](https://github.com/NVlabs/stylegan3) is trained on [the Cedar platform](https://docs.alliancecan.ca/wiki/Cedar). This model supports [three configs](https://github.com/NVlabs/stylegan3/blob/main/docs/configs.md): 
+StyleGAN3-T (translation equiv.), StyleGAN3-R (translation and rotation equiv.), or StyleGAN2.
+
+Refer to [this documentation](./PrepareImageSetForStyleGAN.md) to prepare the image set for this training.
+
+## Start the training job
+
+Step 1: Use [`rsync`](https://linuxhandbook.com/transfer-files-ssh/) or other commands to transfer transformed Bliss
+images to Cedar
+
+Step 2. Login to the Cedar and fetch stylegan3 source code
+```
+mkdir stylegan3
+cd stylegan3
+git clone https://github.com/NVlabs/stylegan3
+```
+
+Step 3. Creating a zip archive of Bliss images will lead to a better performance
+```
+cd stylegan3
+python dataset_tool.py --source=../bliss_single_chars_final --dest=../datasets/bliss-256x256.zip
+```
+
+Step 4: Submit Job
+
+* Copy [requirements.txt](../jobs/stylegan3/requirements.txt) to stylegan3 source code root directory.
+
+* Copy [job_stylegan3.sh](../jobs/stylegan3/job_stylegan3.sh) to the `scratch/' directory in your home directory
+
+* Submit the job
+
+```
+cd ~/scratch
+sbatch job_stylegan3.sh
+```
+
+Use `sq` to check the status of the job. Use `scancel` to cancel a running job.
+
+## The training result
+
+The Bliss images were first trained using `stylegan3-r` config (translation and rotation equiv.). This job had to be
+cancelled after running 2.5 days because the cluster the job was running on is a shared resource that was waited by
+another team. Before the cancellation, the job had generated some training result that can be found at
+[this repository](https://github.com/cindyli/bliss-data/tree/main/styleGAN/styleGAN-training-results/stylegan3-r).
+
+`reals.png` is a collection of real Bliss symobles
+
+`fakes*.png` are random image grids exported from the training loop at regular intervals.
+
+`training_options.json` contains training options used for this round of training.
+
+`metric-fid50k_full.jsonl` logs the result and records` FID evaluated by the training loop for every export.
+
+## Use a trained model to generate images
+
+Looking at `metric-fid50k_full.jsonl`, `network-snapshot-004160.pkl` model has the lowest FID value. Run this command
+to submit a job to generate an image using this model:
+
+```
+sbatch job_stylegan3-t_gen_images.sh
+```
+
+`job_stylegan3-t_gen_images.sh` can be located [here](../jobs/stylegan3/job_stylegan3-t_gen_images.sh).
+
+## Conclusion
+
+After studying the training results and images generated by a trained model. Generated images are not good enough
+for brainstorming new Bliss symbols. The decision is not to continue explore this model.

jobs/README.md

@@ -12,5 +12,5 @@ The [stylegan3](./stylegan3) directory contains:
 * `job_stylegan3.sh` is the job script submitted in [the Cedar platform](https://docs.alliancecan.ca/wiki/Cedar)
 to perform the training.
 
-Refer to the [documentation](../docs/StyleGANModelTraining.md) about how to prepare the image set and train
-the model.
+Refer to the [documentation](../docs/TrainStyleGAN3MOdel.md) about how to train this model, training results and
+the conclusion about how useful it is.

jobs/stylegan3/job_stylegan3-t_gen_images.sh

@@ -0,0 +1,37 @@
+#!/bin/bash
+#SBATCH --job-name=stylegan3-t-gen-images
+#SBATCH --time 10-00:00
+#SBATCH --nodes=1
+#SBATCH --gpus-per-node=v100l:1
+#SBATCH --mem=128G
+#SBATCH --ntasks-per-node=4
+#SBATCH --cpus-per-task=4
+#SBATCH --account=def-whkchun
+#SBATCH --output=%x.o%j
+
+pip install --no-index --upgrade pip
+module load python/3.8.2
+python -V
+
+virtualenv --no-download $SLURM_TMPDIR/env
+source $SLURM_TMPDIR/env/bin/activate
+
+module load scipy-stack/2021a
+module load cuda/11.1.1
+
+echo "Start to check CUDA Toolkit version"
+nvcc -V
+echo "End of checking CUDA Toolkit version"
+
+echo "Start to check gcc version"
+which gcc
+echo "End of checking gcc version"
+
+pip install -r /home/cindyli/stylegan3/stylegan3/requirements.txt
+
+pip list
+
+export CUDA_LAUNCH_BLOCKING=1
+
+echo "Hello from job $SLURM_JOB_ID on nodes $SLURM_JOB_NODELIST."
+python /home/cindyli/stylegan3/stylegan3/gen_images.py --outdir=/home/cindyli/stylegan3/out-stylegan3-t --trunc=1 --seeds=2 --network=/home/cindyli/stylegan3/training-runs/00016-stylegan3-t-bliss-256x256-gpus1-batch32-gamma2/network-snapshot-004160.pkl

notebooks/README.md

@@ -15,3 +15,5 @@ section 1 needs to be filled in before running. If using other platforms, this s
 credential verification for that platform.
 
 `texture_inversion_dockerfile.yml`: the docker environment used for running this training.
+
+**Conclusion**: Texture inversion model is not useful in generating new Bliss symbols for brainstorming.

requirements.txt

@@ -1,2 +1,4 @@
-Pillow
 flake8
+
+# Used by all image processing scripts
+Pillow