Contour-based Multidirectional Prediction for Intra Coding

• Applicant: Professor Dr.-Ing. Jörn Ostermann
• Subject Area: Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering
• Term: from 2018 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 397975900

Intra coding is an essential part of all modern image and video codecs. It is used at the beginning of a transmission, for random access into ongoing transmissions, for error resilience, for changing the data rate in video on demand applications as well as for coding of new content in video sequences. Different to inter coding, intra coding does not use any previously transmitted information for improving the coding efficiency. Therefore, the data rate for intra coding is 10-fold to 100-fold compared to inter coding for the same picture quality. This is the reason for conducting further research in the area of spatial prediction for intra coding.Known predictions methods for intra coding do not perform well in case of a signal rich in contours. This is especially true in case of several or non-linear contours. Therefore methods for the prediction of signals containing several contours were developed. Based on the literature and our own work it is known that it is useful to have separate methods for the extrapolation of contours and for the prediction of pixel values. None of the known methods provides a model for contours that can describe linear and non-linear contours as well as contours where the change of direction is predictable. However, a generic model is desirable for proper prediction of these complex contours in order to minimize the data rate. Known methods for the prediction of pixel values are not sufficiently precise, too compute intensive or unsuitable as a prediction signal for future image frames.The goal of this project is the development of more efficient intra coding methods for image coding as well as video coding by combining traditional video coding methods with contour extrapolation and machine learning methods for pixel value prediction. We will model contours in previously coded neighboring blocks using a Gauss process. The model will be appropriate for any contour, be it linear, non-linear or direction changing. Using the Gauss process, we will extrapolate the contours into the areas of the image to be coded next. Using the previously coded image signal and the extrapolated contours as a reference, we will use a CNN to predict the pixel values. The integration of these methods into image and video codecs will show that the coding efficiency of image and video coding can be improved.

• DFG Programme: Research Grants