We research the synergistic combination of neural networks and physics simulations. We develop neural prediction methods that translate the physics fields into a compressed representation and predict the state of the system over time in the latent space. To increase robustness of long-term predictions we train an end-to-end network and maximize control by latent space subdivision of input quantities.
Artistically controlling the shape, motion and appearance of simulations is essential for providing directability for physics. We use neural networks to transfer styles from arbitrary input images to 3D smoke simulations by reformulating the problem as a transport-based optimization. Our methods generate coherent structures spatially and temporally, and enable effects such as stylization of smoke and liquids, multi-fluid stylization and color transfer.
Authoring physics-based smoke animation is a long-standing problem in visual effects production. We develop new sketching metaphors for the efficient control of fluid animations while exploring the use of data-driven approaches to reduce the computation time for interactive prototyping of animations.
In collaboration with the Institute for Advanced Study at the Technical University of Munich we pursue research on capturing real fluids and computing accurate tomography reconstructions. The real-world captured data will be connected to physics simulations by deep learning methods that allow a high-quality mapping between the different data sources.
The SPH concept is increasingly popular in a large variety of application areas that range from entertainment technologies to engineering. We have developed various methods for increasing stability, efficiency and versatility of SPH simulations, including predictor-corrector methods for pressure computation and a machine learning based real-time fluid solver.
In this multi-investigator project funded by the Botnar Research Centre for Child Health, the University of Basel and our group aim to optimize the surgical treatment of cleft lip and palate with the use of machine learning algorithms, smartphone images and 3D printed orthopedic plates.