Social-media Photo Appeal (SoPhoAppeal)

• Applicant: Professor Dr.-Ing. Alexander Raake
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering
• Term: from 2020 to 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 437543412

• Final Report Year: 2023

Final Report Abstract

The project covers three main aspects, namely, the dataset, the features, and the prediction models, which are all publicly shared and open source. The dataset includes 1061 high-resolution images from six sources and has furthermore ratings of four conducted subjective studies, meta-data, likes/views, saliency/depth/segmentation maps, features, and SoA prediction model values included. This is unique and new, in comparison to other published datasets, which usually have less data included. The work can be seen as an extension of the work of this project towards the analysis of image appeal and quality considering AI-generated images. For this, too, the dataset and evaluation is shared, and the extracted image features are based on the features developed in this project. This extension is also related to the work, where AI-based algorithms are used for image upscaling and compared to each other. Furthermore, the AVRate Voyager framework provided in the course of the project can be used for various online tests, as it is also shown. The data, models, and feature extraction code are shared and can be used for follow-up work. The COVID-19 pandemic resulted in a change of the time plan, however, it also emphasized that crowd-tests may be a good alternative for image and video assessment, even considering higher resolutions.

Publications

• AVrate Voyager: an open source online testing platform. 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), 1-6. IEEE.
  Goring, Steve; Ramachandra, Rao Rakesh Rao; Fremerey, Stephan & Raake, Alexander
  
• Rule of Thirds and Simplicity for Image Aesthetics using Deep Neural Networks. 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), 1-6. IEEE.
  Goring, Steve & Raake, Alexander
  
• Analysis of Appeal for Realistic AI-Generated Photos. IEEE Access, 11, 38999-39012.
  Göring, Steve; Ramachandra, Rao Rakesh Rao; Merten, Rasmus & Raake, Alexander
  
• Appeal and quality assessment for AI-generated images. 2023 15th International Conference on Quality of Multimedia Experience (QoMEX), 115-118. IEEE.
  Göring, Steve; Ramachandra, Rao Rakesh Rao; Merten, Rasmus & Raake, Alexander
  
• AVT-VQDB-UHD-1-Appeal: A UHD-1/4K Open Dataset for Video Quality and Appeal Assessment Using Modern Video Codecs. 2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP), 1-6. IEEE.
  Rao, Rakesh Rao Ramachandra; Göring, Steve; Elmeligy, Bassem & Raake, Alexander
  
• DNN-based Photography Rule Prediction using Photo Tags. 2023 15th International Conference on Quality of Multimedia Experience (QoMEX), 83-86. IEEE.
  Göring, Steve; Merten, Rasmus & Raake, Alexander
  
• Image Appeal Revisited: Analysis, New Dataset, and Prediction Models. IEEE Access, 11, 69563-69585.
  Göring, Steve & Raake, Alexander
  
• Quality assessment of higher resolution images and videos with remote testing. Quality and User Experience, 8(1).
  Göring, Steve; Rao, Rakesh Rao Ramachandra & Raake, Alexander
  
• Appeal prediction for AI up-scaled Images. 2024 International Symposium on Multimedia (ISM), 55-62. IEEE.
  Goring, Steve; Merten, Rasmus & Raake, Alexander