Skip to content

Commit

Permalink
added docs
Browse files Browse the repository at this point in the history
  • Loading branch information
saandeepa93 committed Jul 7, 2024
1 parent 45231a6 commit fcaeac0
Show file tree
Hide file tree
Showing 25 changed files with 13 additions and 81 deletions.
94 changes: 13 additions & 81 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -138,27 +138,19 @@ <h2 class="title is-3">Abstract</h2>
deforming scene using photos/videos captured casually from mobile phones.
</p>
<p>
Our approach augments neural radiance fields
(NeRF) by optimizing an
additional continuous volumetric deformation field that warps each observed point into a
canonical 5D NeRF.
We observe that these NeRF-like deformation fields are prone to local minima, and
propose a coarse-to-fine optimization method for coordinate-based models that allows for
more robust optimization.
By adapting principles from geometry processing and physical simulation to NeRF-like
models, we propose an elastic regularization of the deformation field that further
improves robustness.
</p>
<p>
We show that <span class="dnerf">Nerfies</span> can turn casually captured selfie
photos/videos into deformable NeRF
models that allow for photorealistic renderings of the subject from arbitrary
viewpoints, which we dub <i>"nerfies"</i>. We evaluate our method by collecting data
using a
rig with two mobile phones that take time-synchronized photos, yielding train/validation
images of the same pose at different viewpoints. We show that our method faithfully
reconstructs non-rigidly deforming scenes and reproduces unseen views with high
fidelity.
Identifying Out-of-distribution (OOD) data is becoming increasingly critical as the real-world applications
of deep learning methods expand. Post-hoc methods modify softmax scores fine-tuned on outlier data or leverage
intermediate feature layers to identify distinctive patterns between In-Distribution (ID) and OOD samples.
Other methods focus on employing diverse OOD samples to learn discrepancies between ID and OOD.
These techniques, however, are typically dependent on the quality of the outlier samples assumed.
Density-based methods explicitly model class-conditioned distributions but this requires long training
time or retraining the classifier. To tackle these issues, we introduce \textit{FlowCon}, a new density-based
OOD detection technique. Our main innovation lies in efficiently combining the properties of normalizing
flow with supervised contrastive learning, ensuring robust representation learning with tractable density estimation.
Empirical evaluation shows the enhanced performance of our method across common vision datasets such as
CIFAR-10 and CIFAR-100 pretrained on ResNet18 and WideResNet classifiers. We also perform quantitative
analysis using likelihood plots and qualitative visualization using UMAP embeddings and demonstrate the
robustness of the proposed method under various OOD contexts. Code will be open-sourced post decision.
</p>
</div>
</div>
Expand Down Expand Up @@ -186,66 +178,6 @@ <h2 class="title is-3">Abstract</h2>
<!--/ Matting. -->

<!-- Animation. -->
<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Animation</h2>

<!-- Interpolating. -->
<h3 class="title is-4">Interpolating states</h3>
<div class="content has-text-justified">
<p>
We can also animate the scene by interpolating the deformation latent codes of two input
frames. Use the slider here to linearly interpolate between the left frame and the right
frame.
</p>
</div>
<div class="columns is-vcentered interpolation-panel">
<div class="column is-3 has-text-centered">
<img src="./static/images/interpolate_start.jpg"
class="interpolation-image"
alt="Interpolate start reference image."/>
<p>Start Frame</p>
</div>
<div class="column interpolation-video-column">
<div id="interpolation-image-wrapper">
Loading...
</div>
<input class="slider is-fullwidth is-large is-info"
id="interpolation-slider"
step="1" min="0" max="100" value="0" type="range">
</div>
<div class="column is-3 has-text-centered">
<img src="./static/images/interpolate_end.jpg"
class="interpolation-image"
alt="Interpolation end reference image."/>
<p class="is-bold">End Frame</p>
</div>
</div>
<br/>
<!--/ Interpolating. -->

<!-- Re-rendering. -->
<h3 class="title is-4">Re-rendering the input video</h3>
<div class="content has-text-justified">
<p>
Using <span class="dnerf">Nerfies</span>, you can re-render a video from a novel
viewpoint such as a stabilized camera by playing back the training deformations.
</p>
</div>
<div class="content has-text-centered">
<video id="replay-video"
controls
muted
preload
playsinline
width="75%">
<source src="./static/videos/replay.mp4"
type="video/mp4">
</video>
</div>
<!--/ Re-rendering. -->

</div>
</div>
<!--/ Animation. -->

Expand Down
Binary file added docs/static/images/figures/far_near.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/images/figures/ll_cifar10.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/images/figures/ll_cifar100.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/images/figures/main_res.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/images/figures/near.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/images/figures/umap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/static/videos/._Icon
Binary file not shown.
Binary file removed docs/static/videos/blueshirt.mp4
Binary file not shown.
Binary file removed docs/static/videos/book.mp4
Binary file not shown.
Binary file removed docs/static/videos/chair-tp.mp4
Binary file not shown.
Binary file removed docs/static/videos/coffee.mp4
Binary file not shown.
Binary file removed docs/static/videos/dollyzoom-depth.mp4
Binary file not shown.
Binary file removed docs/static/videos/dollyzoom-stacked.mp4
Binary file not shown.
Binary file removed docs/static/videos/dollyzoom.mp4
Binary file not shown.
Binary file removed docs/static/videos/fullbody.mp4
Binary file not shown.
Binary file removed docs/static/videos/mask-nobg.mp4
Binary file not shown.
Binary file removed docs/static/videos/mask.mp4
Binary file not shown.
Binary file removed docs/static/videos/matting.mp4
Binary file not shown.
Binary file removed docs/static/videos/replay.mp4
Binary file not shown.
Binary file removed docs/static/videos/shiba.mp4
Binary file not shown.
Binary file removed docs/static/videos/steve.mp4
Binary file not shown.
Binary file removed docs/static/videos/teaser.mp4
Binary file not shown.
Binary file removed docs/static/videos/toby.mp4
Binary file not shown.
Binary file removed docs/static/videos/toby2.mp4
Binary file not shown.

0 comments on commit fcaeac0

Please sign in to comment.