filename     : Bri21a.pdf
entry        : inproceedings
conference   : SIGGRAPH Asia 2021, Tokyo, Japan, 14-17 December, 2021
pages        : 239:1-239:13
year         : 2021
month        : 12
title        : Neural Frame Interpolation for Rendered Content
subtitle     :
author       : Karlis Martins Briedis, Abdelaziz Djelouah, Mark Meyer, Ian McGonigal, Markus Gross, and Christopher Schroers
booktitle    : ACM Transactions on Graphics (TOG) - SIGGRAPH Asia 2021 Conference Proceedings
ISSN/ISBN    : 0730-0301
editor       :
publisher    : Association for Computing Machinery
publ.place   :
volume       : 40
issue        : 6
language     : English
keywords     : frame interpolation, motion estimation, deep learning
abstract     : The demand for creating rendered content continues to drastically grow. As it often is extremely computationally expensive and thus costly to render high-quality computer-generated images, there is a high incentive to reduce this computational burden. Recent advances in learning-based frame interpolation methods have shown exciting progress but still have not achieved the production-level quality which would be required to render fewer pixels and achieve savings in rendering times and costs. Therefore, in this paper we propose a method specifically targeted to achieve high-quality frame interpolation for rendered content. In this setting, we assume that we have full input for every n-th frame in addition to auxiliary feature buffers that are cheap to evaluate (e.g. depth, normals, albedo) for every frame. We propose solutions for leveraging such auxiliary features to obtain better motion estimates, more accurate occlusion handling, and to correctly reconstruct non-linear motion between keyframes. With this, our method is able to significantly push the state-of-the-art in frame interpolation for rendered content and we are able to obtain production-level quality results.