SimAnim@ETHZ

BRINGING A SMILE TO YOUR FACE

Together with Align Technology and the University of Geneva, we push the frontiers of the medical metaverse by developing data-driven methods for 3D patient modeling and physics-based simulation of soft tissue changes arising from orthodontic treatments.

FOR THE YOUNGEST PATIENTS

Our vision is to enable a stress-reduced treatment for every child born with a cleft - worldwide. In collaboration with USB and supported by BRCCH, we use graphics and AI to automatically compute a plate for the presurgical treatment of infant patients.

3D HEAD MODELS
FROM DATA

Creating 3D head models is extremely difficult and manual, hindering integration into day-to-day workflows. We develop data-driven methods to automate the process and to infer missing data. We apply this to infants and adults, targeted at applications in medical fields.

AI DRIVEN PHYSICAL FACES

With Disney Research | Studios we develop a methods to inject physics into face animation technologies. By treating the face as an actuated soft body, we can leverage differentiable physics and neural networks to animate expressions.

INTERACTIVE CHARACTERS

Digital characters are increasingly used in entertainment, education or customer support. We develop methods for data-driven animation synthesis to enable expressive and engaging conversational stylized characters.

ART-DIRECTED FLUIDS

Artistically controlling the shape, motion and appearance of fluid simulations pose major challenges in visual effects production. We use differentiable simulation and learning for style transfer, keyframe matching and artistic deformation.

3D FLUID RECONSTRUCTION

In collaboration with TUM, we enable the reconstruction of volumetric flow motions from sparse and single-view input videos. Our algorithm uses transport constraints, differentiable rendering and neural networks for robust end-to-end reconstructions.

AFFECTIVE STATES

Identification of affective states enables the design of emotionally sentient systems. We have developed methods for affective state prediction based on camera recordings, low cost mobile biosensors, handwriting data, and smartphone touch and sensor data.

PERSONALIZED LEARNING

We have developed intelligent training environments for learning spelling and mathematics. Our data-driven student model enables a personalized presentation of the content. We leverage ML for analytics and visualization tools targeted at teachers and experts.

Research