Spaces:
Running
Running
<html> | |
<head> | |
<meta charset="utf-8"> | |
<meta name="description" | |
content="Deformable Neural Radiance Fields creates free-viewpoint portraits (nerfies) from casually captured videos."> | |
<meta name="keywords" content="Nerfies, D-NeRF, NeRF"> | |
<meta name="viewport" content="width=device-width, initial-scale=1"> | |
<title>Nerfies: Deformable Neural Radiance Fields</title> | |
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" | |
rel="stylesheet"> | |
<link rel="stylesheet" href="./static/css/bulma.min.css"> | |
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css"> | |
<link rel="stylesheet" href="./static/css/bulma-slider.min.css"> | |
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css"> | |
<link rel="stylesheet" | |
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> | |
<link rel="stylesheet" href="./static/css/index.css"> | |
<link rel="icon" href="./static/images/favicon.svg"> | |
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script> | |
<script defer src="./static/js/fontawesome.all.min.js"></script> | |
<script src="./static/js/bulma-carousel.min.js"></script> | |
<script src="./static/js/bulma-slider.min.js"></script> | |
<script src="./static/js/index.js"></script> | |
</head> | |
<body> | |
<section class="hero teaser"> | |
<div class="container is-max-desktop"> | |
<div class="hero-body"> | |
<script | |
type="module" | |
src="https://gradio.s3-us-west-2.amazonaws.com/5.20.0/gradio.js" | |
></script> | |
<gradio-app src="https://mobvoi-offical-spark-tts.hf.space"></gradio-app> | |
<section class="section"> | |
<div class="container is-max-desktop"> | |
<!-- Abstract. --> | |
<div class="columns is-centered has-text-centered"> | |
<div class="column is-four-fifths"> | |
<h2 class="title is-3">Abstract</h2> | |
<div class="content has-text-justified"> | |
<p> | |
<slot> | |
</slot> | |
</p> | |
</div> | |
</div> | |
</div> | |
<!--/ Abstract. --> | |
<!-- Paper video. --> | |
<div class="columns is-centered has-text-centered"> | |
<div class="column is-four-fifths"> | |
<h2 class="title is-3">Video</h2> | |
<div class="publication-video"> | |
<iframe src="https://www.youtube.com/embed/MrKrnHhk8IA?rel=0&showinfo=0" | |
frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe> | |
</div> | |
</div> | |
</div> | |
<!--/ Paper video. --> | |
</div> | |
</section> | |
<section class="section"> | |
<div class="container is-max-desktop"> | |
<div class="columns is-centered"> | |
<!-- Visual Effects. --> | |
<div class="column"> | |
<div class="content"> | |
<h2 class="title is-3">Visual Effects</h2> | |
<p> | |
<slot id=2 name="second"></slot> | |
</p> | |
<iframe | |
src="https://mobvoi-offical-spark-tts.hf.space" | |
frameborder="0" | |
width="850" | |
height="450" | |
></iframe> | |
</div> | |
</div> | |
<!--/ Visual Effects. --> | |
<!-- Matting. --> | |
<div class="column"> | |
<h2 class="title is-3">Matting</h2> | |
<div class="columns is-centered"> | |
<div class="column content"> | |
<p> | |
As a byproduct of our method, we can also solve the matting problem by ignoring | |
samples that fall outside of a bounding box during rendering. | |
</p> | |
<video id="matting-video" controls playsinline height="100%"> | |
<source src="./static/videos/matting.mp4" | |
type="video/mp4"> | |
</video> | |
</div> | |
</div> | |
</div> | |
</div> | |
<!--/ Matting. --> | |
<!-- Animation. --> | |
<div class="columns is-centered"> | |
<div class="column is-full-width"> | |
<h2 class="title is-3">Animation</h2> | |
<!-- Interpolating. --> | |
<h3 class="title is-4">Interpolating states</h3> | |
<div class="content has-text-justified"> | |
<p> | |
We can also animate the scene by interpolating the deformation latent codes of two input | |
frames. Use the slider here to linearly interpolate between the left frame and the right | |
frame. | |
</p> | |
</div> | |
<div class="columns is-vcentered interpolation-panel"> | |
<div class="column is-3 has-text-centered"> | |
<img src="./static/images/interpolate_start.jpg" | |
class="interpolation-image" | |
alt="Interpolate start reference image."/> | |
<p>Start Frame</p> | |
</div> | |
<div class="column interpolation-video-column"> | |
<div id="interpolation-image-wrapper"> | |
Loading... | |
</div> | |
<input class="slider is-fullwidth is-large is-info" | |
id="interpolation-slider" | |
step="1" min="0" max="100" value="0" type="range"> | |
</div> | |
<div class="column is-3 has-text-centered"> | |
<img src="./static/images/interpolate_end.jpg" | |
class="interpolation-image" | |
alt="Interpolation end reference image."/> | |
<p class="is-bold">End Frame</p> | |
</div> | |
</div> | |
<br/> | |
<!--/ Interpolating. --> | |
<!-- Re-rendering. --> | |
<h3 class="title is-4">Re-rendering the input video</h3> | |
<div class="content has-text-justified"> | |
<p> | |
Using <span class="dnerf">Nerfies</span>, you can re-render a video from a novel | |
viewpoint such as a stabilized camera by playing back the training deformations. | |
</p> | |
</div> | |
<div class="content has-text-centered"> | |
<video id="replay-video" | |
controls | |
muted | |
preload | |
playsinline | |
width="75%"> | |
<source src="./static/videos/replay.mp4" | |
type="video/mp4"> | |
</video> | |
</div> | |
<!--/ Re-rendering. --> | |
</div> | |
</div> | |
<!--/ Animation. --> | |
<!-- Concurrent Work. --> | |
<div class="columns is-centered"> | |
<div class="column is-full-width"> | |
<h2 class="title is-3">Related Links</h2> | |
<div class="content has-text-justified"> | |
<p> | |
There's a lot of excellent work that was introduced around the same time as ours. | |
</p> | |
<p> | |
<a href="https://arxiv.org/abs/2104.09125" target="_blank">Progressive Encoding for Neural Optimization</a> introduces an idea similar to our windowed position encoding for coarse-to-fine optimization. | |
</p> | |
<p> | |
<a href="https://www.albertpumarola.com/research/D-NeRF/index.html" target="_blank">D-NeRF</a> and <a href="https://gvv.mpi-inf.mpg.de/projects/nonrigid_nerf/" target="_blank">NR-NeRF</a> | |
both use deformation fields to model non-rigid scenes. | |
</p> | |
<p> | |
Some works model videos with a NeRF by directly modulating the density, such as <a href="https://video-nerf.github.io/" target="_blank">Video-NeRF</a>, <a href="https://www.cs.cornell.edu/~zl548/NSFF/" target="_blank">NSFF</a>, and <a href="https://neural-3d-video.github.io/" target="_blank">DyNeRF</a> | |
</p> | |
<p> | |
There are probably many more by the time you are reading this. Check out <a href="https://dellaert.github.io/NeRF/" target="_blank">Frank Dellart's survey on recent NeRF papers</a>, and <a href="https://github.com/yenchenlin/awesome-NeRF" target="_blank">Yen-Chen Lin's curated list of NeRF papers</a>. | |
</p> | |
</div> | |
</div> | |
</div> | |
<!--/ Concurrent Work. --> | |
</div> | |
</section> | |
<section class="section" id="BibTeX"> | |
<div class="container is-max-desktop content"> | |
<h2 class="title">BibTeX</h2> | |
<pre><code>@article{park2021nerfies, | |
author = {Park, Keunhong and Sinha, Utkarsh and Barron, Jonathan T. and Bouaziz, Sofien and Goldman, Dan B and Seitz, Steven M. and Martin-Brualla, Ricardo}, | |
title = {Nerfies: Deformable Neural Radiance Fields}, | |
journal = {ICCV}, | |
year = {2021}, | |
}</code></pre> | |
</div> | |
</section> | |
<footer class="footer"> | |
<div class="container"> | |
<div class="content has-text-centered"> | |
<a class="icon-link" target="_blank" | |
href="./static/videos/nerfies_paper.pdf"> | |
<i class="fas fa-file-pdf"></i> | |
</a> | |
<a class="icon-link" href="https://github.com/keunhong" target="_blank" class="external-link" disabled> | |
<i class="fab fa-github"></i> | |
</a> | |
</div> | |
<div class="columns is-centered"> | |
<div class="column is-8"> | |
<div class="content"> | |
<p> | |
This website is licensed under a <a rel="license" target="_blank" | |
href="http://creativecommons.org/licenses/by-sa/4.0/">Creative | |
Commons Attribution-ShareAlike 4.0 International License</a>. | |
</p> | |
<p> | |
This means you are free to borrow the <a target="_blank" | |
href="https://github.com/nerfies/nerfies.github.io">source code</a> of this website, | |
we just ask that you link back to this page in the footer. | |
Please remember to remove the analytics code included in the header of the website which | |
you do not want on your website. | |
</p> | |
</div> | |
</div> | |
</div> | |
</div> | |
</footer> | |
</body> | |
</html> | |