filename     : Xu24a.pdf
entry        : inproceedings
conference   : CVPR 2024, Seattle, US, 17-21 June, 2024
pages        : 2457-2467
year         : 2024
month        : June
title        : Artist-Friendly Relightable and Animatable Neural Heads
subtitle     :
author       : Yingyan Xu, Prashanth Chandran, Sebastian Weiss, Markus Gross, Gaspard Zoss, Derek Bradley
booktitle    : Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
ISSN/ISBN    : 
editor       :
publisher    : IEEE
publ.place   :
volume       :
issue        :
language     : English
keywords     : neural avatars, relighting, animation
abstract     : An increasingly common approach for creating photo-realistic digital avatars is through the use of volumetric neural fields. The original neural radiance field (NeRF) allowed for impressive novel view synthesis of static heads when trained on a set of multi-view images and follow up methods showed that these neural representations can be extended to dynamic avatars. Recently new variants also surpassed the usual drawback of baked-in illumination in neural representations showing that static neural avatars can be relit in any environment. In this work we simultaneously tackle both the motion and illumination problem proposing a new method for relightable and animatable neural heads. Our method builds on a proven dynamic avatar approach based on a mixture of volumetric primitives combined with a recently-proposed lightweight hardware setup for relightable neural fields and includes a novel architecture that allows relighting dynamic neural avatars performing unseen expressions in any environment even with nearfield illumination and viewpoints.