High-Quality Single-Shot Capture of Facial Geometry
T. Beeler, B. Bickel, P. Beardsley, R. Sumner, M. Gross
Proceedings of ACM SIGGRAPH (Los Angeles, USA, July 25-29, 2010), ACM Transactions on Graphics, vol. 29, no. 3, pp. 40:1-40:9
Abstract
This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are sub-millimeter accurate and commensurate with those from state-of-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography. Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.
Sample Dataset
We provide a sample dataset for research purposes.
What the archive contains
The archive contains the captured image data, calibrated cameras as
well as the reconstructed mesh. The file structure is:
cameras
cam1.cam
cam2.cam
...
data
cam1
img_xxx.cr2
img_xxx.jpg
cam2
...
...
DISCLAIMER
result
mesh.ply
The file formats
Images
The images are provided both as jpg
as well as canon raw (cr2)
format.
The raw images can be read for example with LibRaw.
Reconstructed Geometry
The format of the reconstructed geometry is either obj or ply. Both formats
contain
vertex positions and normals as well as indices. The ply format further
contains per vertex colors. Both formats
can be read and converted with Meshlab.
Cameras
The camera format is our own. It consists of two lines; a header and
the actual parameters. The header describes the content of the
parameters, possible values are:
The human face is very personal and we decided thus not to publish the
data online. On the other hand, high quality reconstruction data is
very valuable to many researchers. As a compromise we offer to send the
data directly to approved researchers. To request the data, please send
an email
to dbeeler at inf dotethz dot ch
stating
your name, title or position, and institution or affiliation
your intended use of the images and/or reconstructed geometry
a statement saying that you accept the following terms of
licensing (please copy the licensing text into your email):
The rights to copy, distribute, and use the 3D computer
models and
image data
(henceforth called "data") you are being given access to are under the
control of Markus Gross, director of the Computer Graphics Lab, ETH
Zurich. You are hereby given permission to copy this
data in electronic or hardcopy form for your own scientific use and to
distribute it for scientific use to colleagues within your research
group.
Inclusion of rendered images or video made from this data in a
scholarly
publication (printed or electronic) is also permitted. In this case,
credit
must be given to the publication: *High-Quality Single-Shot Capture of
Facial Geometry*.
However, the data may not be included in the electronic version of a
publication, nor placed on the Internet. These restrictions apply to
any
representations (other than images or video) derived from the data,
including
but not limited to simplifications, remeshings, and the fitting of
smooth
surfaces. The making of physical replicas this
data is also prohibited, and the data may not be distributed to
students - also not in
connection with a class. For any other use, including distribution
outside
your research group, written permission is required from Markus Gross.
Any
commercial use also requires written permission from Markus Gross.
Commercial use includes but is not limited to sale of the data,
derivatives,
replicas, images, or video, inclusion in a product for sale, or
inclusion in
advertisements (printed or electronic), on commercially-oriented web
sites, or
in trade shows.
Inappropriate use
Please remember that faces are of very personal
nature. Keep your renderings and other uses of the data
in good taste. Don't put the faces in degrading or tasteless context
and don't simulate nasty things happening to them
(like
breaking, exploding, melting, etc.). Choose another model for these
sorts of
experiments. Also, exercise reasonable caution to prevent the data from
wandering beyond your research group.
Commercial use
Please contact dbeeler at infdot
ethz dot ch if you are interested in using the data and/or the
system/algorithms commercially.