Contour-based Multidirectional Prediction for Intra Coding
Final Report Abstract
The data rate for video transmission increases faster than the channel capacity. Therefore, new and improved video coding algorithms are necessary. A modern video codec is a hybrid codec that is a combination of temporal prediction and transform coding of the prediction error. For prediction, intra and inter methods are available. In this work, we propose two methods for improving intra prediction. The first contribution is a statistical contour model for modelling and extrapolating contours that are detected in a reference area. We use a gaussian process for modeling. The expected shape of contours is considered using a squared exponential kernel as covariance function of the prior Gaussian process. The posterior gaussian process is derived from the prior gaussian process by optimizing hyper parameters of the covariance function of each contour. For extrapolation, we formulate a multivariate gaussian distribution. The second contribution of this work is a method for sample prediction using a neural network. The neural network uses as input the neighboring samples in the reference area, the results of the contour modelling as well as the results of the contour extrapolation. It predicts the sample values of the block to be coded. The contours are required in order to achieve a stable prediction. The neural network was designed using an autoencoder architecture. The loss function during training was SATD. The coding efficiency of HEVC extended by the above methods increases by up to 5%. Averaged across 55 test sequences coded using an all-intra configuration, coding efficiency (in terms of BD-rate) increased by 0.54 % for high data rates and by 1.0% for low data rate. Compared to our own works prior to this project, the gains are about 0.2% BD-rate. The work program was executed as planed and all relevant milestones were achieved. Based on the recommendation of one of the evaluators of the project proposal, we were able to transfer our results in the area of machine learning for intra video coding to other areas of video coding. This resulted in an additional publication on inter coding using neural networks.
Publications
- Non‐linear contour‐based multidirectional intra coding. APSIPA Transactions on Signal and Information Processing, 7(11), 2018
Thorsten Laude, Jan Tumbrägel, Marco Munderloh und Jörn Ostermann
(See online at https://doi.org/10.1017/ATSIP.2018.14) - „A Comparison of JEM and AV1 with HEVC: Coding Tools, Coding Efficiency and Complexity“. Proceedings of the IEEE Picture Coding Symposium (PCS). Juni 2018
Thorsten Laude, Yeremia Gunawan Adhisantoso, Jan Voges, Marco Munderloh und Jörn Ostermann
(See online at https://doi.org/10.1109/PCS.2018.8456291) - HEVC Inter Coding using Deep Recurrent Neural Networks and Artificial Reference Pictures. Picture Coding Symposium (PCS), 2019
Thorsten Laude, Felix Haub und Jörn Ostermann
(See online at https://doi.org/10.1109/PCS48520.2019.8954497) - „A Comprehensive Video Codec Comparison“. APSIPA Transactions on Signal and Information Processing 8 (Nov. 2019)
Thorsten Laude, Yeremia Gunawan Adhisantoso, Jan Voges, Marco Munderloh und Jörn Ostermann
(See online at https://doi.org/10.1017/ATSIP.2019.23) - „Contour‐based Intra Coding Using Gaussian Processes and Neural Networks“. Proceedings of the IEEE Picture Coding Symposium (PCS). Juli 2021
Thorsten Laude and Jörn Ostermann
(See online at https://doi.org/10.1109/PCS50896.2021.9477500)
