Computer Graphics Laboratory ETH Zurich

ETH

Novel-View Synthesis of Outdoor Sport Events Using an Adaptive View-Dependent Geometry

M. Germann, T. Popa, R. Keiser, R. Ziegler, M. Gross

Proceedings of Eurographics (Cagliari, Italy, May 13-18, 2012), Computer Graphics Forum, vol. 31, no. 2, pp. 325-333
[Abstract] [BibTeX] [PDF]

Abstract

We propose a novel fully automatic method for novel-viewpoint synthesis. Our method robustly handles multi-camera setups featuring wide-baselines in an uncontrolled environment. In a first step, robust and sparse point correspondences are found based on an extension of the Daisy features~\cite{Tola09}. These correspondences together with back-projection errors are used to drive a novel adaptive coarse to fine reconstruction method, allowing to approximate detailed geometry while avoiding an extreme triangle count. To render the scene from arbitrary viewpoints we use a view-dependent blending of color information in combination with a view-dependent geometry morph. The view-dependent geometry compensates for misalignments caused by calibration errors. We demonstrate that our method works well under arbitrary lighting conditions with as little as two cameras featuring wide-baselines. The footage taken from real sports broadcast events contains fine geometric structures, which result in nice novel-viewpoint renderings despite of the low resolution in the images.

@article{Ger10,
    title = {Novel-View Synthesis of Outdoor Sport Events Using an Adaptive View-Dependent Geometry},
    author = {Marcel Germann and Tiberiu Popa and Richard Keiser and Remo Ziegler and Markus Gross},
    journal = {Comput. Graphics Forum (Proc. Eurographics)},
    pages = {???--???},
    number = {2},
    volume = {31},
    year = {2012}
}
[Download BibTeX]

Overview

Figure 1: Overview of the algorithm: (a)-(e) The adaptive reconstruction. (f) The view-dependent rendering.

Our algorithm computes a 2.5D reconstruction for every camera as a base camera (Fig. 1 (a) -(d)). In a first step, an initial triangulation is set (Fig. 1 (a)). This is followed by iterations between depth optimization of the triangle vertices (Fig. 1 (b)) and subdivision steps. The depth optimization uses feature point matches from the base camera to other cameras and rough calibrations. In the subdivision, a back-projection error is evaluated to determine if a triangle could not be fit onto the geometry and needs to be subdivided. After a refinement of the depth values according to back-projection errors (Fig. 1 (d)), the 2.5D reconstructions of different base cameras are merged into a final 3D geometry.

Instead of using only projective textures with view-dependent blending, we propose a view-dependent geometry morph (Fig. 1 (f)). It is able to reduce texture missalignments caused by calibration errors. The 3D geometry morph is computed according to feature matches which are not on the epipolar lines of each other. These non-epipolar matches are used to determine the error of the calibration locally.

Results

Our method is able to reconstruct the geomety in difficult outdoor setups with wide baselines, low resolution coverage, weak calibrations, and motion blur. In the paper and the video we show results that were generated with only two standard TV cameras as a source. Currently, there is no temporal coherence used. Nevertheless, we show results of sequences that demonstrate the ability to process many frames and only contain a minor flickering.

Figure 2: Rendering result from a novel view generated by using only two standard TV cameras as source.

The view-dependent geometry significantly improves the result compared to standard projective texturing. The ghosting artifacts (duplication of object parts) are reduced to a minimum but the geometry and thus the rendered images still preserve the interpolation of the original camera views.

Downloads

Download Paper
[PDF]