Analysis and VLSI Implementation of EWA Rendering for Real-time HD Video Applications
P. Greisen, M. Schaffner, S. Heinzle, M. Runo, A. Smolic, A. Burg, H. Kaeslin, M. GrossIEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 11, 2012, pp. 1577-1589
Abstract
Non-linear image warping or image resampling is a necessary step in many current and upcoming video applications such as video retargeting, stereoscopic 3D mapping, and multiview synthesis. The challenges for real-time resampling include image quality but also available energy and computational power of the employed device. In this work, we employ an elliptical-weighted average (EWA) rendering approach to 2D image resampling. We extend the classical EWA framework for increased visual quality and provide a VLSI architecture for efficient view rendering. The resulting architecture is able to render high-quality video sequences in real-time targeted for lowpower applications in end-user display devices.Overview
This work consists of two parts: an analysis and optimization of the EWA splatting algorithm and a corresponding VLSI architecture for real-time non-linear warping. First, we extend the traditional EWA splatting algorithm by showing how to optimally chose the filter parameters and by providing an adaptive scheme that optimizes the tradeoff between blurring and aliasing. Also, to practically deal with the infinite impulse response (IIR) of EWA filters, we show how to select cut-off points in the rendered target space. Secondly, to provide a low-power, low-cost, and small size solution, we propose a VLSI architecture of the derived EWA splatting algorithm for real-time, high-resolution non-linear warping. To cope with the large memory bandwidth requirements of EWA splatting, we propose a two-level caching architecture that significantly reduces the required memory bandwidth. Further, we investigate various number formats for EWA splatting. Finally, we provide area and performance results for a fabricated design in a 180 nm CMOS process.
Results
EWA splatting is a promising technique for current and nextgeneration HD video applications such as video retargeting, disparity mapping, and multi-view synthesis. Setting the Gaussian filter variances in an adaptive way greatly improves rendering quality. Thus, with the proposed adaptive strategy, we are able to render high-quality images without aliasing or excessive blurring.
Furthermore, we show that EWA rendering can be efficiently implemented into a VLSI circuit, which would be targeted for end-user display integration. The proposed VLSI architecture for real-time EWA splatting provides high-quality results using fixed-precision number formats. Multi-level accumulation significantly reduces the necessary memory bandwidth to the external frame buffer.
