Multi-exposure fusion is a critical issue in computer vision. Additionally, this technique can be adopt in smart phone to demonstrate the image with high lighting quality. However, the original author didn't release the official implementation. In this repository, we try to re-produce the idea of DeepFuse [1], and fuse the under-exposure image and over-exposure image with appropriate manner.
The above image shows the training result. The most left sub-figure is the under-exposure image. The second sub-figure is the over-exposure image. The third one is the rendered result, and the most right figure is the ground truth which is compute by MEF-SSIM loss concept. As you can see, the rough information of dark region and light region can be both remained. The following image is another example.
You should notice that this is not the official implementation. There are several different between this repository and the paper:
- Since the dataset that author used cannot be obtained, we use HDR-Eye dataset [2] which can also deal with multiple exposure fusion problem.
- Rather use 64*64 patch size, we set the patch size as 256*256.
- We only train for 20 epochs. (30000 iteration for each epoch)
- The calculation of y^hat is different. The detail can be found in here.
The detail of parameters can be found here. You can just simply use the command to train the DeepFuse:
python3 train.py --folder ./SunnerDataset/HDREyeDataset/images/Bracketed_images --batch_size 8 --epoch 15000
Or you can download the pre-trained model here. Furthermore, inference with two image:
python3 inference.py --image1 <UNDER_EXPOSURE_IMG_PATH> --image2 <OVER_EXPOSURE_IMG_PATH> --model train_result/model/latest.pth --res result.png
After we check for several machine, we found that the program might get stuck at cv2.cvtColor function. We infer the reason is that the OpenCV cannot perfectly embed in the multiprocess mechanism which is provided by Pytorch. As the result, we assign num_worker as zero here to avoid the issue. If your machine doesn't encounter this issue, you can add the number to accelerate the loading process.
[1] K. R. Prabhakar, V. S. Srikar, and R. V. Babu. Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. In 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, pages 4724–4732, 2017.
[2] H. Nemoto, P. Korshunov, P. Hanhart, and T. Ebrahimi. Visual attention in ldr and hdr images. In 9th International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), number EPFL-CONF-203873, 2015.