ArcGAN: “Generative Adversarial Networks for 3D Architectural Image Generation”

Figure: Recovered 3D shape and rotation&relighting effects using ArcGAN.

ArcGAN: “Generative Adversarial Networks for 3D Architectural Image Generation”
Drashty Ranpara, Raghul V, Arun Mozhi Devan, Samiksha Shukla

In this repository, we present ArcGAN, which reconstructs the 3D shape of an image using 2D image GANs in an unsupervised manner. Our method does not rely on mannual annotations or external 3D models, yet it achieves high-quality 3D reconstruction, object rotation, and relighting effects.

Requirements

• python>=3.6
• pytorch=1.1 or 1.2
• neural_renderer

  pip install neural_renderer_pytorch  # or follow the guidance at https://github.com/elliottwu/unsup3d

• mmcv

  pip install mmcv

• other dependencies

  conda install -c conda-forge scikit-image matplotlib opencv pyyaml tensorboardX

Dataset and pre-trained weights

To download dataset and pre-trained weights, simply run:

sh scripts/download.sh

Training

https://github.com/DrashtySoni/ArcGAN

Example: training on lsun church images:

!python run.py --launcher none --config configs/church.yml 2>&1 | tee results/architecture/log.txt

This would run on 4 GPUs by default. You can view the results at results/architecture/images or Tensorboard.

Note:
- For church, the quality of StyleGAN2 samples vary a lot, thus our approach may not produce good result on every sample. The downloaded dataset contains examples of good samples.

Acknowledgement

Some part of the code is borrowed from Unsup3d and StyleGAN2.