diff --git a/README.md b/README.md index 369f220..34d0baf 100644 --- a/README.md +++ b/README.md @@ -39,11 +39,11 @@ where `esrgan/config.yml` is a path to the [config](config.yml) file. ## Results Some examples of work of ESRGAN model trained on [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K) dataset: -| LR
(low resolution) | ESRGAN
(ours) | HR
(high resolution) | -|:---:|:---:|:---:| -| | | | -| | | | -| | | | +| LR
(low resolution) | ESRGAN
(original) | ESRGAN
(ours) | HR
(high resolution) | +|:---:|:---:|:---:|:---:| +| | | | | +| | | | | +| | | | | ## Documentation Full documentation for the project is available at https://esrgan.readthedocs.io/ diff --git a/config.yml b/config.yml index 3f52e97..2f81c83 100644 --- a/config.yml +++ b/config.yml @@ -94,7 +94,7 @@ stages: - _target_: albumentations.Normalize mean: 0 std: 1 - - _target_: albumentations.ToTensorV2 + - _target_: albumentations.pytorch.ToTensorV2 additional_targets: real_image: image low_resolution_image_key: image diff --git a/docs/_static/0802.png b/docs/_static/0802.png new file mode 100644 index 0000000..62c2782 Binary files /dev/null and b/docs/_static/0802.png differ diff --git a/docs/_static/0802hr.png b/docs/_static/0802hr.png new file mode 100644 index 0000000..33fc726 Binary files /dev/null and b/docs/_static/0802hr.png differ diff --git a/docs/_static/0802lr.png b/docs/_static/0802lr.png new file mode 100644 index 0000000..4940a59 Binary files /dev/null and b/docs/_static/0802lr.png differ diff --git a/docs/_static/0802sr.png b/docs/_static/0802sr.png new file mode 100644 index 0000000..26d7659 Binary files /dev/null and b/docs/_static/0802sr.png differ diff --git a/docs/_static/0805.png b/docs/_static/0805.png new file mode 100644 index 0000000..790bc8f Binary files /dev/null and b/docs/_static/0805.png differ diff --git a/docs/_static/0805hr.png b/docs/_static/0805hr.png new file mode 100644 index 0000000..9ff1710 Binary files /dev/null and b/docs/_static/0805hr.png differ diff --git a/docs/_static/0805lr.png b/docs/_static/0805lr.png new file mode 100644 index 0000000..72df4e2 Binary files /dev/null and b/docs/_static/0805lr.png differ diff --git a/docs/_static/0805sr.png b/docs/_static/0805sr.png new file mode 100644 index 0000000..de0bec0 Binary files /dev/null and b/docs/_static/0805sr.png differ diff --git a/docs/_static/0811.png b/docs/_static/0811.png new file mode 100644 index 0000000..5e69b5f Binary files /dev/null and b/docs/_static/0811.png differ diff --git a/docs/_static/0811hr.png b/docs/_static/0811hr.png new file mode 100644 index 0000000..a59dcaf Binary files /dev/null and b/docs/_static/0811hr.png differ diff --git a/docs/_static/0811lr.png b/docs/_static/0811lr.png new file mode 100644 index 0000000..c0b530a Binary files /dev/null and b/docs/_static/0811lr.png differ diff --git a/docs/_static/0811sr.png b/docs/_static/0811sr.png new file mode 100644 index 0000000..bb3f48b Binary files /dev/null and b/docs/_static/0811sr.png differ diff --git a/docs/_static/0815.png b/docs/_static/0815.png new file mode 100644 index 0000000..60e980f Binary files /dev/null and b/docs/_static/0815.png differ diff --git a/docs/_static/0815hr.png b/docs/_static/0815hr.png new file mode 100644 index 0000000..e3180c7 Binary files /dev/null and b/docs/_static/0815hr.png differ diff --git a/docs/_static/0815lr.png b/docs/_static/0815lr.png new file mode 100644 index 0000000..f8ff8ab Binary files /dev/null and b/docs/_static/0815lr.png differ diff --git a/docs/_static/0815sr.png b/docs/_static/0815sr.png new file mode 100644 index 0000000..f5e336c Binary files /dev/null and b/docs/_static/0815sr.png differ diff --git a/docs/_static/0829.png b/docs/_static/0829.png new file mode 100644 index 0000000..06b6fca Binary files /dev/null and b/docs/_static/0829.png differ diff --git a/docs/_static/0829hr.png b/docs/_static/0829hr.png new file mode 100644 index 0000000..da8a794 Binary files /dev/null and b/docs/_static/0829hr.png differ diff --git a/docs/_static/0829lr.png b/docs/_static/0829lr.png new file mode 100644 index 0000000..f4a4888 Binary files /dev/null and b/docs/_static/0829lr.png differ diff --git a/docs/_static/0829sr.png b/docs/_static/0829sr.png new file mode 100644 index 0000000..7163d3c Binary files /dev/null and b/docs/_static/0829sr.png differ diff --git a/docs/_static/0845.png b/docs/_static/0845.png new file mode 100644 index 0000000..9ad1990 Binary files /dev/null and b/docs/_static/0845.png differ diff --git a/docs/_static/0845hr.png b/docs/_static/0845hr.png new file mode 100644 index 0000000..787ad66 Binary files /dev/null and b/docs/_static/0845hr.png differ diff --git a/docs/_static/0845lr.png b/docs/_static/0845lr.png new file mode 100644 index 0000000..b700346 Binary files /dev/null and b/docs/_static/0845lr.png differ diff --git a/docs/_static/0845sr.png b/docs/_static/0845sr.png new file mode 100644 index 0000000..7e08a8f Binary files /dev/null and b/docs/_static/0845sr.png differ diff --git a/docs/_static/0853.png b/docs/_static/0853.png new file mode 100644 index 0000000..5c903b4 Binary files /dev/null and b/docs/_static/0853.png differ diff --git a/docs/_static/0853hr.png b/docs/_static/0853hr.png new file mode 100644 index 0000000..a731d66 Binary files /dev/null and b/docs/_static/0853hr.png differ diff --git a/docs/_static/0853lr.png b/docs/_static/0853lr.png new file mode 100644 index 0000000..ac51265 Binary files /dev/null and b/docs/_static/0853lr.png differ diff --git a/docs/_static/0853sr.png b/docs/_static/0853sr.png new file mode 100644 index 0000000..dc2d402 Binary files /dev/null and b/docs/_static/0853sr.png differ diff --git a/docs/_static/0857.png b/docs/_static/0857.png new file mode 100644 index 0000000..174c366 Binary files /dev/null and b/docs/_static/0857.png differ diff --git a/docs/_static/0857hr.png b/docs/_static/0857hr.png new file mode 100644 index 0000000..776255f Binary files /dev/null and b/docs/_static/0857hr.png differ diff --git a/docs/_static/0857lr.png b/docs/_static/0857lr.png new file mode 100644 index 0000000..86bfff7 Binary files /dev/null and b/docs/_static/0857lr.png differ diff --git a/docs/_static/0857sr.png b/docs/_static/0857sr.png new file mode 100644 index 0000000..34bff77 Binary files /dev/null and b/docs/_static/0857sr.png differ diff --git a/docs/_static/0886.png b/docs/_static/0886.png new file mode 100644 index 0000000..e357ea2 Binary files /dev/null and b/docs/_static/0886.png differ diff --git a/docs/_static/0886hr.png b/docs/_static/0886hr.png new file mode 100644 index 0000000..f1c4733 Binary files /dev/null and b/docs/_static/0886hr.png differ diff --git a/docs/_static/0886lr.png b/docs/_static/0886lr.png new file mode 100644 index 0000000..8941cb6 Binary files /dev/null and b/docs/_static/0886lr.png differ diff --git a/docs/_static/0886sr.png b/docs/_static/0886sr.png new file mode 100644 index 0000000..37e2142 Binary files /dev/null and b/docs/_static/0886sr.png differ diff --git a/docs/_static/0887.png b/docs/_static/0887.png new file mode 100644 index 0000000..2babb09 Binary files /dev/null and b/docs/_static/0887.png differ diff --git a/docs/_static/0887hr.png b/docs/_static/0887hr.png new file mode 100644 index 0000000..0a56b2b Binary files /dev/null and b/docs/_static/0887hr.png differ diff --git a/docs/_static/0887lr.png b/docs/_static/0887lr.png new file mode 100644 index 0000000..c4059f0 Binary files /dev/null and b/docs/_static/0887lr.png differ diff --git a/docs/_static/0887sr.png b/docs/_static/0887sr.png new file mode 100644 index 0000000..d46c1e8 Binary files /dev/null and b/docs/_static/0887sr.png differ diff --git a/docs/_static/fish_crop_hr.png b/docs/_static/fish_crop_hr.png deleted file mode 100644 index 7b97b3d..0000000 Binary files a/docs/_static/fish_crop_hr.png and /dev/null differ diff --git a/docs/_static/fish_crop_lr.png b/docs/_static/fish_crop_lr.png deleted file mode 100644 index 502a7dc..0000000 Binary files a/docs/_static/fish_crop_lr.png and /dev/null differ diff --git a/docs/_static/fish_crop_sr_x4.png b/docs/_static/fish_crop_sr_x4.png deleted file mode 100644 index cfe4e2c..0000000 Binary files a/docs/_static/fish_crop_sr_x4.png and /dev/null differ diff --git a/docs/_static/sq_crop_hr.png b/docs/_static/sq_crop_hr.png deleted file mode 100644 index cf6a7c4..0000000 Binary files a/docs/_static/sq_crop_hr.png and /dev/null differ diff --git a/docs/_static/sq_crop_lr.png b/docs/_static/sq_crop_lr.png deleted file mode 100644 index 98aefd5..0000000 Binary files a/docs/_static/sq_crop_lr.png and /dev/null differ diff --git a/docs/_static/sq_crop_sr_x4.png b/docs/_static/sq_crop_sr_x4.png deleted file mode 100644 index 17af9af..0000000 Binary files a/docs/_static/sq_crop_sr_x4.png and /dev/null differ diff --git a/docs/_static/wf_crop_hr.png b/docs/_static/wf_crop_hr.png deleted file mode 100644 index 052cc22..0000000 Binary files a/docs/_static/wf_crop_hr.png and /dev/null differ diff --git a/docs/_static/wf_crop_lr.png b/docs/_static/wf_crop_lr.png deleted file mode 100644 index 26bfb17..0000000 Binary files a/docs/_static/wf_crop_lr.png and /dev/null differ diff --git a/docs/_static/wf_crop_sr_x4.png b/docs/_static/wf_crop_sr_x4.png deleted file mode 100644 index bdd2a40..0000000 Binary files a/docs/_static/wf_crop_sr_x4.png and /dev/null differ diff --git a/docs/conf.py b/docs/conf.py index 583629b..0cfeaad 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -30,7 +30,7 @@ def _get_project_meta(): pkg_meta = _get_project_meta() project = str(pkg_meta["name"]) -copyright = "2020, Emerline, Inc. and its affiliates." # noqa: WPS125 +copyright = "2022, Emerline, Inc. All Rights Reserved." # noqa: WPS125 author = "Emerline" # The short X.Y version @@ -101,3 +101,5 @@ def _get_project_meta(): html_static_path = ["_static"] html_short_title = project + +html_scaled_image_link = False diff --git a/docs/index.rst b/docs/index.rst index 4a7f169..04bee3f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -62,26 +62,28 @@ Results Some examples of work of ESRGAN model trained on `DIV2K `__ dataset: .. table:: - :widths: 33 33 33 - - +---------------------------------------+------------------------------------------+---------------------------------------+ - | .. centered:: LR (low resolution) | .. centered:: ESRGAN (ours) | .. centered:: HR (high resolution) | - +=======================================+==========================================+=======================================+ - | .. image:: ./_static/sq_crop_lr.png | .. image:: ./_static/sq_crop_sr_x4.png | .. image:: ./_static/sq_crop_hr.png | - | :width: 128px | :width: 128px | :width: 128px | - | :height: 128px | :height: 128px | :height: 128px | - | :align: center | :align: center | :align: center | - +---------------------------------------+------------------------------------------+---------------------------------------+ - | .. image:: ./_static/wf_crop_lr.png | .. image:: ./_static/wf_crop_sr_x4.png | .. image:: ./_static/wf_crop_hr.png | - | :width: 128px | :width: 128px | :width: 128px | - | :height: 128px | :height: 128px | :height: 128px | - | :align: center | :align: center | :align: center | - +---------------------------------------+------------------------------------------+---------------------------------------+ - | .. image:: ./_static/fish_crop_lr.png | .. image:: ./_static/fish_crop_sr_x4.png | .. image:: ./_static/fish_crop_hr.png | - | :width: 128px | :width: 128px | :width: 128px | - | :height: 128px | :height: 128px | :height: 128px | - | :align: center | :align: center | :align: center | - +---------------------------------------+------------------------------------------+---------------------------------------+ + :widths: 25 25 25 25 + + +-----------------------------------+---------------------------------+-------------------------------+------------------------------------+ + | .. centered:: LR (low resolution) | .. centered:: `ESRGAN`_ | .. centered:: ESRGAN (ours) | .. centered:: HR (high resolution) | + +===================================+=================================+===============================+====================================+ + | .. image:: ./_static/0853lr.png | .. image:: ./_static/0853sr.png | .. image:: ./_static/0853.png | .. image:: ./_static/0853hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+---------------------------------+-------------------------------+------------------------------------+ + | .. image:: ./_static/0857lr.png | .. image:: ./_static/0857sr.png | .. image:: ./_static/0857.png | .. image:: ./_static/0857hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+---------------------------------+-------------------------------+------------------------------------+ + | .. image:: ./_static/0887lr.png | .. image:: ./_static/0887sr.png | .. image:: ./_static/0887.png | .. image:: ./_static/0887hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+---------------------------------+-------------------------------+------------------------------------+ + +.. _ESRGAN: https://github.com/xinntao/ESRGAN GitHub diff --git a/docs/pages/esrgan.rst b/docs/pages/esrgan.rst index b435435..790d5e0 100644 --- a/docs/pages/esrgan.rst +++ b/docs/pages/esrgan.rst @@ -3,7 +3,7 @@ ESRGAN ====== -Network Architecture +Network architecture ==================== .. image:: ../_static/architecture.png @@ -19,7 +19,68 @@ Main modifications in the structure of generator G, discriminator D, and trainin 4. perceptual loss on features before activation. -Qualitative Results +Some results +============ + +.. table:: + :widths: 25 25 25 25 + + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. centered:: LR (low resolution) | .. centered:: ESRGAN `\[1\]`_ | .. centered:: ESRGAN (ours) | .. centered:: HR (high resolution) | + +===================================+==================================+================================+====================================+ + | .. image:: ../_static/0802lr.png | .. image:: ../_static/0802sr.png | .. image:: ../_static/0802.png | .. image:: ../_static/0802hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0805lr.png | .. image:: ../_static/0805sr.png | .. image:: ../_static/0805.png | .. image:: ../_static/0805hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0811lr.png | .. image:: ../_static/0811sr.png | .. image:: ../_static/0811.png | .. image:: ../_static/0811hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0815lr.png | .. image:: ../_static/0815sr.png | .. image:: ../_static/0815.png | .. image:: ../_static/0815hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0829lr.png | .. image:: ../_static/0829sr.png | .. image:: ../_static/0829.png | .. image:: ../_static/0829hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0845lr.png | .. image:: ../_static/0845sr.png | .. image:: ../_static/0845.png | .. image:: ../_static/0845hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0853lr.png | .. image:: ../_static/0853sr.png | .. image:: ../_static/0853.png | .. image:: ../_static/0853hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0857lr.png | .. image:: ../_static/0857sr.png | .. image:: ../_static/0857.png | .. image:: ../_static/0857hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0886lr.png | .. image:: ../_static/0886sr.png | .. image:: ../_static/0886.png | .. image:: ../_static/0886hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + | .. image:: ../_static/0887lr.png | .. image:: ../_static/0887sr.png | .. image:: ../_static/0887.png | .. image:: ../_static/0887hr.png | + | :width: 128px | :width: 128px | :width: 128px | :width: 128px | + | :height: 128px | :height: 128px | :height: 128px | :height: 128px | + | :align: center | :align: center | :align: center | :align: center | + +-----------------------------------+----------------------------------+--------------------------------+------------------------------------+ + + +Qualitative results =================== PSNR (evaluated on the Y channel) and the perceptual index used in the PIRM-SR challenge @@ -35,7 +96,7 @@ Qualitative Results :width: 100% -Ablation Study +Ablation study ============== Overall visual comparisons for showing the effects of each component in ESRGAN.