Occlusion-aware temporal frame interpolation in a highly scalable video coding setting

APSIPA Transactions on Signal and Information Processing

Dominic Rüfenacht¹  David Taubman¹  Reji Mathew¹ 

[1] University of New South Wales

关键词: Temporal frame interpolation;    Temporal frame upsampling;    Geometrically consistent prediction;    Occlusion-handling;    Motion discontinuity modeling;    Bidirectional hierarchical anchoring of motion fields;   

DOI  :  10.1017/ATSIP.2016.8

学科分类：计算机科学（综合）

来源: Cambridge University Press

【 摘 要 】

We recently proposed a bidirectional hierarchical anchoring (BIHA) of motion fields for highly scalable video coding. The BIHA scheme employs piecewise-smooth motion fields, and uses breakpoints to signal motion discontinuities. In this paper, we show how the fundamental building block of the BIHA scheme can be used to perform bidirectional, occlusion-aware temporal frame interpolation (BOA-TFI). From a “parent” motion field between two reference frames, we use information about motion discontinuities to compose motion fields from both reference frames to the target frame; these then get inverted so that they can be used to predict the target frame. During the motion inversion process, we compute a reliable occlusion mask, which is used to guide the bidirectional motion-compensated prediction of the target frame. The scheme can be used in any state-of-the-art codec, but is most beneficial if used in conjunction with a highly scalable video coder which employs piecewise-smooth motion fields with motion discontinuities. We evaluate the proposed BOA-TFI scheme on a large variety of natural and challenging computer-generated sequences, and our results compare favorably to state-of-the-art TFI methods.

【 授权许可】

Unknown