Skip to content

Commit

Permalink
add demo link
Browse files Browse the repository at this point in the history
  • Loading branch information
Alvin Liu authored and Alvin Liu committed Oct 12, 2023
1 parent e91d33b commit d9d1bc1
Show file tree
Hide file tree
Showing 4 changed files with 48 additions and 6 deletions.
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion

[Xian Liu](https://alvinliu0.github.io/), [Jian Ren](https://alanspike.github.io/), [Aliaksandr Siarohin](https://aliaksandrsiarohin.github.io/aliaksandr-siarohin-website/), [Ivan Skorokhodov](https://universome.github.io/), [Yanyu Li](https://scholar.google.com/citations?user=XUj8koUAAAAJ&hl=en), [Dahua Lin](http://dahua.site/), [Xihui Liu](https://xh-liu.github.io/), [Ziwei Liu](https://liuziwei7.github.io/), [Sergey Tulyakov](http://www.stulyakov.com/).
[Xian Liu](https://alvinliu0.github.io/)<sup>1,2</sup>, [Jian Ren](https://alanspike.github.io/)<sup>1</sup>, [Aliaksandr Siarohin](https://aliaksandrsiarohin.github.io/aliaksandr-siarohin-website/)<sup>1</sup>, [Ivan Skorokhodov](https://universome.github.io/)<sup>1</sup>, [Yanyu Li](https://scholar.google.com/citations?user=XUj8koUAAAAJ&hl=en)<sup>1</sup>,
[Dahua Lin](http://dahua.site/)<sup>2</sup>, [Xihui Liu](https://xh-liu.github.io/)<sup>3</sup>, [Ziwei Liu](https://liuziwei7.github.io/)<sup>4</sup>, [Sergey Tulyakov](http://www.stulyakov.com/)<sup>1</sup>.
<sup>1</sup>Snap Inc.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<sup>2</sup>CUHK&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<sup>3</sup>HKU&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<sup>4</sup>NTU

### [Project](https://snap-research.github.io/HyperHuman/) | [Paper](https://arxiv.org/pdf/xxxx)
### [Project](https://snap-research.github.io/HyperHuman/) | [Paper](https://snap-research.github.io/HyperHuman/content/hyperhuman.pdf) | [arXiv](https://arxiv.org/pdf/xxxx) | [Short Demo (3min)](https://www.youtube.com/watch?v=eRPZW1pwxog) | [Long Demo (10min)](https://www.youtube.com/watch?v=CxGfbwZOcyU)

Despite significant advances in large-scale text-to-image models, achieving hyper-realistic human image generation remains a desirable yet unsolved task. Existing models like Stable Diffusion and DALL·E 2 tend to generate human images with incoherent parts or unnatural poses. To tackle these challenges, our key insight is that human image is inherently structural over multiple granularities, from the coarse-level body skeleton to fine-grained spatial geometry. Therefore, capturing such correlations between the explicit appearance and latent structure in one model is essential to generate coherent and natural human images. To this end, we propose a unified framework, **HyperHuman**, that generates in-the-wild human images of high realism and diverse layouts. Specifically, **1)** we first build a large-scale human-centric dataset, named *HumanVerse*, which consists of 340M images with comprehensive annotations like human pose, depth, and surface normal. **2)** Next, we propose a *Latent Structural Diffusion Model* that simultaneously denoises the depth and surface normal along with the synthesized RGB image. Our model enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network, where each branch in the model complements to each other with both structural awareness and textural richness. **3)** Finally, to further boost the visual quality, we propose a *Structure-Guided Refiner* to compose the predicted conditions for more detailed generation of higher resolution. Extensive experiments demonstrate that our framework yields the state-of-the-art performance, generating hyper-realistic human images under diverse scenarios.

Expand Down
Binary file added content/.DS_Store
Binary file not shown.
Binary file modified content/hyperhuman.pdf
Binary file not shown.
48 changes: 44 additions & 4 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,24 @@
</a>
</span>
<!-- Video Link. -->
<!-- <span class="link-block">
<a href="https://www.youtube.com/watch?v=Fvenkw7yeok"
<span class="link-block">
<a href="https://www.youtube.com/watch?v=eRPZW1pwxog"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-youtube"></i>
</span>
<span>Short Demo (3min)</span>
</a>
</span>
<span class="link-block">
<a href="https://www.youtube.com/watch?v=CxGfbwZOcyU"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-youtube"></i>
</span>
<span>Demo</span>
<span>Long Demo (10min)</span>
</a>
</span> -->
</span>
<!-- Code Link. -->
<span class="link-block">
<a href="https://github.com/snap-research/HyperHuman"
Expand Down Expand Up @@ -228,6 +237,37 @@

<!-- === Result Section Starts === -->
<!-- === Result Section Ends === -->
<div class="section">
<div class="title">Short Demo Video (3min)</div>
<div class="body">

We present a short demo video, mostly with visualization results and a very quick overview of our framework.

<!-- Adjust the frame size based on the demo (EVERY project differs). -->
<div style="position: relative; padding-top: 50%; margin: 20pt 0; text-align: center;">
<iframe src="https://www.youtube.com/watch?v=eRPZW1pwxog" frameborder=0
style="position: absolute; top: 2.5%; left: 2.5%; width: 95%; height: 100%;"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen></iframe>
</div>
</div>
</div>

<div class="section">
<div class="title">Long Demo Video (10min)</div>
<div class="body">

We present a long demo video with detailed elaborations on the motivations and framework designs.

<!-- Adjust the frame size based on the demo (EVERY project differs). -->
<div style="position: relative; padding-top: 50%; margin: 20pt 0; text-align: center;">
<iframe src="https://www.youtube.com/watch?v=CxGfbwZOcyU" frameborder=0
style="position: absolute; top: 2.5%; left: 2.5%; width: 95%; height: 100%;"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen></iframe>
</div>
</div>
</div>

<!-- === Overview Section Starts === -->
<div class="section">
Expand Down

0 comments on commit d9d1bc1

Please sign in to comment.