One paper has been accepted to MDPI Sensors Journal (IF 3.847).
Title: Joint Video Super-Resolution and Frame Interpolation via Permutation Invariance
Authors: Jinsoo Choi, Tae-Hyun Oh
We propose a joint super resolution (SR) and frame interpolation framework that can 1 perform both spatial and temporal super resolution. We identify performance variation according to 2 permutation of inputs in video super-resolution and video frame interpolation. We postulate that 3 favorable features extracted from multiple frames should be consistent regardless of input order if 4 the features are optimally complementary for respective frames. With this motivation, we propose a 5 permutation invariant deep architecture that makes use of the multi-frame SR principles by virtue 6 of our order (permutation) invariant network. Specifically, given two adjacent frames, our model 7 employs a permutation invariant convolutional neural network module to extract “complementary” 8 feature representations facilitating both the SR and temporal interpolation tasks. We demonstrate the 9 effectiveness of our end-to-end joint method against various combinations of the competing SR and 10 frame interpolation methods on challenging video datasets, and thereby we verify our hypothesis.