High-Quality Single-Shot Capture of Facial Geometry

T. Beeler, B. Bickel, P. Beardsley, R. Sumner, M. Gross

Proceedings of ACM SIGGRAPH (Los Angeles, USA, July 25-29, 2010), ACM Transactions on Graphics, vol. 29, no. 3, pp. 40:1-40:9

Abstract

This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are sub-millimeter accurate and commensurate with those from state-of-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography. Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.

Sample Dataset

We provide a sample dataset for research purposes.

What the archive contains
The archive contains the captured image data, calibrated cameras as well as the reconstructed mesh. The file structure is:

cameras

cam1.cam
cam2.cam
...

data

cam1

img_xxx.cr2
img_xxx.jpg

cam2

DISCLAIMER
result

mesh.ply

The file formats

Images
The images are provided both as jpg as well as canon raw (cr2) format. The raw images can be read for example with LibRaw.

Reconstructed Geometry
The format of the reconstructed geometry is either obj or ply. Both formats contain vertex positions and normals as well as indices. The ply format further contains per vertex colors. Both formats can be read and converted with Meshlab.

Cameras
The camera format is our own. It consists of two lines; a header and the actual parameters. The header describes the content of the parameters, possible values are:

Property		Description
name		Name of the camera
fx,fy		Focal length
cx,cy		Principal point
alpha		Skew
nx,ny		Image size
k1..k5		Distortion parameters as described by Bouguet
tx,ty,tz		Extrinsic translation
rx,ry,rz		Extrinsic rotation (given in Rodrigues notation)
zn,zf		Near and far planes of the working volume

Obtaining the data

The human face is very personal and we decided thus not to publish the data online. On the other hand, high quality reconstruction data is very valuable to many researchers. As a compromise we offer to send the data directly to approved researchers. To request the data, please send an email derek dot bradley at disneyresearch dot com stating

your name, title or position, and institution or affiliation
your intended use of the images and/or reconstructed geometry
a statement saying that you accept the following terms of licensing (please copy the licensing text into your email):
The rights to copy, distribute, and use the 3D computer models and image data (henceforth called "data") you are being given access to are under the control of Markus Gross, director of the Computer Graphics Lab, ETH Zurich. You are hereby given permission to copy this data in electronic or hardcopy form for your own scientific use and to distribute it for scientific use to colleagues within your research group. Inclusion of rendered images or video made from this data in a scholarly publication (printed or electronic) is also permitted. In this case, credit must be given to the publication: *High-Quality Single-Shot Capture of Facial Geometry*. However, the data may not be included in the electronic version of a publication, nor placed on the Internet. These restrictions apply to any representations (other than images or video) derived from the data, including but not limited to simplifications, remeshings, and the fitting of smooth surfaces. The making of physical replicas this data is also prohibited, and the data may not be distributed to students - also not in connection with a class. For any other use, including distribution outside your research group, written permission is required from Markus Gross. Any commercial use also requires written permission from Markus Gross. Commercial use includes but is not limited to sale of the data, derivatives, replicas, images, or video, inclusion in a product for sale, or inclusion in advertisements (printed or electronic), on commercially-oriented web sites, or in trade shows.

Inappropriate use

Please remember that faces are of very personal nature. Keep your renderings and other uses of the data in good taste. Don't put the faces in degrading or tasteless context and don't simulate nasty things happening to them (like breaking, exploding, melting, etc.). Choose another model for these sorts of experiments. Also, exercise reasonable caution to prevent the data from wandering beyond your research group.