Coupled 3D Reconstruction of Sparse Facial Hair and Skin
T. Beeler, B. Bickel, G. Noris, S. Marschner, P. Beardsley, R. Sumner, M. Gross
Proceedings of ACM SIGGRAPH (Los Angeles, USA, August 5-9, 2012), ACM Transactions on Graphics, vol. 31, no. 4, pp. 117:1-117:10 [Abstract]
Although facial hair plays an important role in individual expression, facial-hair reconstruction is not addressed by current face-capture systems. Our research addresses this limitation with an algorithm that treats hair and skin surface capture together in a coupled fashion so that a high-quality representation of hair fibers as well as the underlying skin surface can be reconstructed. We propose a passive, camera-based system that is robust against arbitrary motion since all data is acquired within the time period of a single exposure. Our reconstruction algorithm detects and traces hairs in the captured images and reconstructs them in 3D using a multi-view stereo approach. Our coupled skin-reconstruction algorithm uses information about the detected hairs to deliver a skin surface that lies underneath all hairs irrespective of occlusions. In dense regions like eyebrows, we employ a hair-synthesis method to create hair fibers that plausibly match the image data. We demonstrate our scanning system on a number of individuals and show that it can successfully reconstruct a variety of facial-hair styles together with the underlying skin surface.
We provide a sample dataset for research purposes.
What the archive contains
The archive contains the captured image data, calibrated cameras as
well as the reconstructed facial geometry (episurface) and the facial hair. The facial hair is stored both in a custom format (see below) and explicitly as mesh geometry (ply & obj).
The file structure is:
The file formats
The images are provided both in jpg and cr2 (Canon RAW) format. These are the original images used by our algorithm.
Facial Geometry (Episurface)
The format of the reconstructed facial episurface is ply. The meshes can be read and converted with Meshlab.
The facial hair is provided both explicitly as mesh data with texture and implicitly as polylines in our own .hcn format.
The mesh format is both ply and obj. The meshes can be read and converted with Meshlab. The texture is provided as .png. For some applications, the Y-axis must be flipped for correct display.
Our own .hcn format is a binary format. The following two functions demonstrate how to read the file:
The human face is very personal and we decided thus not to publish the
data online. On the other hand, high quality reconstruction data is
very valuable to many researchers. As a compromise we offer to send the
data directly to approved researchers. To request the data, please send
to dbeeler at inf dotethz dot ch
your name, title or position, and institution or affiliation
your intended use of the images and/or reconstructed geometry
a statement saying that you accept the following terms of
licensing (please copy the licensing text into your email):
The rights to copy, distribute, and use the 3D computer
(henceforth called "data") you are being given access to are under the
control of Markus Gross, director of the Computer Graphics Lab, ETH
Zurich. You are hereby given permission to copy this
data in electronic or hardcopy form for your own scientific use and to
distribute it for scientific use to colleagues within your research
Inclusion of rendered images or video made from this data in a
publication (printed or electronic) is also permitted. In this case,
must be given to the publication: *Coupled 3D Reconstruction of Sparse Facial Hair and Skin*.
However, the data may not be included in the electronic version of a
publication, nor placed on the Internet. These restrictions apply to
representations (other than images or video) derived from the data,
but not limited to simplifications, remeshings, and the fitting of
surfaces. The making of physical replicas this
data is also prohibited, and the data may not be distributed to
students - also not in
connection with a class. For any other use, including distribution
your research group, written permission is required from Markus Gross.
commercial use also requires written permission from Markus Gross.
Commercial use includes but is not limited to sale of the data,
replicas, images, or video, inclusion in a product for sale, or
advertisements (printed or electronic), on commercially-oriented web
in trade shows.
Please remember that faces are of very personal
nature. Keep your renderings and other uses of the data
in good taste. Don't put the faces in degrading or tasteless context
and don't simulate nasty things happening to them
breaking, exploding, melting, etc.). Choose another model for these
experiments. Also, exercise reasonable caution to prevent the data from
wandering beyond your research group.
Please contact dbeeler at infdot
ethz dot ch if you are interested in using the data and/or the