Computer Graphics Laboratory

CGL » Research » Image and Video

Image and Video

Introduction

The goal of our research is to study and develop algorithms for image and video processing, editing, analysis and synthesis. Our focus lies on developing highly efficient algorithms which can be applied to real-world high resolution image and video data.

Topics

Video Segmentation

Video object segmentation is a binary labeling problem aiming to separate foreground object(s) from the background region of a video. A pixel-accurate, spatio-temporal bipartition of the video is instrumental to several applications including, among others, action recognition, object tracking, video summarization, and rotoscoping for video editing. Despite remarkable progress in recent years, video object segmentation still remains a challenging problem and most existing approaches still exhibit too severe limitations in terms of quality and efficiency to be applicable in practical applications, e.g. for processing large datasets, or video post-production and editing in the visual effects industry.


Phase-based Methods for Video

Phase-based methods rely on the assumption that small motions can be encoded in the phase shift of an individual pixel. Due to per-pixel phase modifications this concept allows the design of highly efficient algorithms. Currently, however, the spatial displacement which can be encoded in the phase information with these methods is highly limited. Our research focuses on overcoming these limitations and designing algorithms for various applications such as frame interpolation and view synthesis.


Video-based Rendering

Video-based rendering aims to generate virtual views of a real world scene that was recorded by one or more video cameras. The goal is to achieve as realistic as possible images based on only the camera input, e.g. from standard TV cameras. We developed novel representation and rendering methods that result in images visually not distinguishable from original camera images. Together with our collaborator, LiberoVision AG, we show a successful application for this: virtual replays of sports events.

Light Field Processing

Since its introduction to the computer graphics community the light field has been widely used as an alternative way to represent visual aspects of 3D objects and scenes. Although it has already proven its potential in many areas such as visual effects in movie industries, still many of such tasks are done manually using conventional tools that are not aware of light fields. We pursue developing algorithms and computational tools to automate its processing and enable new paradigms of image and video manipulation. We also aim to extend its use to a broader range of graphics and vision problems. Our research consists of light field acquisition, spatio-temporal manipulation, geometry reconstruction, and 3D rendering, spanning the entire life cycle of light fields.


Stereoscopy

Stereoscopic 3D has gained significant importance in the entertainment industry. However, production of high quality stereoscopic content is still a challenging art that requires mastering the complex interplay of human perception, 3D display properties, and artistic intent. Our research ranges from development of the stereoscopic camera system to computational methods to deal with the difficulty that arises in the course of stereoscopic content creation.


Publications





© 2024 ETH Zurich, Computer Graphics Laboratory