filename : p_Wue04.pdf entry : inproceedings pages : 3-14 year : 2004 month : January title : 3D Video Fragments: Dynamic Point Samples for Real-time Free-Viewpoint Video author : Stephan Würmlin, Edouard Lamboray, Markus Gross booktitle : Computers and Graphics, 28 (1), Special Issue on Coding, Compression and Streaming Techniques for 3D and Multimedia Data ISSN/ISBN : 0097-8493 editor : Stefan Gumhold, Stefan Noll publisher : Elsevier Ltd. publ.place : Amsterdam, The Netherlands language : English keywords : 3D video, distributed graphics systems, inter-frame coding, point based graphics, real-time graphics abstract : We present 3D video fragments, a dynamic point sample framework for real-time free-viewpoint video. By generalizing 2D video pixels towards 3D irregular point samples we combine the simplicity of conventional 2D video processing with the power of more complex polygonal representations for free-viewpoint video. We propose a differential update scheme exploiting the spatio-temporal coherence of the video streams of multiple cameras. Updates are issued by operators such as inserts and deletes accounting for changes in the input video images. The operators from multiple cameras are processed, merged into a 3D video stream and transmitted to a remote site. We also introduce a novel concept for camera control which dynamically selects the set of relevant cameras for reconstruction. Moreover, it adapts to the processing load and rendering platform. Our framework is generic in the sense that it works with any real-time 3D reconstruction method which extracts depth from images. The video renderer displays free-viewpoint videos using an efficient point based splatting scheme and makes use of state-of-the-art vertex and pixel processing hardware for real-time visual processing.