Auto-Adaptive Learning from Weak Feedback for Interactive Lecture Translation

• Applicant: Professor Dr. Stefan Riezler
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
• Term: from 2017 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 326904228

• Final Report Year: 2022

Final Report Abstract

Auto-adaptive learning from weak feedback in the area of machine translation describes a mutually beneficial learning cycle where a human user is supported by a machine translation system, and where human feedback is used directly as weak signal for machine learning. The main advantage of this framework is that weak feedback for machine translations is available more easily and in larger quantities from humans than professional translations for supervised learing. The project investigated algorithms for interactive machine learning from weak feedback, and showed successful applications to speech recognition and translation. The algorithms presented in the project have been successfully applied in academic and commercial settings. Publications describing the work of the project appeared in the most prestigious conferences in the fields of speech and natural language processing. Tangible outcomes of the project are widely used datasets for speech recognition and speech translation, and open-source toolkits for neural machine translation.

Publications

• (2018). A reinforcement learning approach to interactive-predictive neural machine translation. In Proceedings of the 21st Annual Conference of the European Association for Machine Translation (EAMT), Alicante, Spain
  Lam, T. K., Kreutzer, J., and Riezler, S.
  
• (2018). Reliability and learnability of human bandit feedback for sequence-to-sequence reinforcement learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), Melbourne, Australia
  Kreutzer, J., Uyheng, J., and Riezler, S.
  
• (2019). Interactive-predictive neural machine translation through reinforcement and imitation. In Proceedings of the Machine Translation Summit (MTSUMMIT XVII), Dublin, Ireland
  Lam, T. K., Schamoni, S., and Riezler, S.
  
• (2019). Joey NMT: A minimalist NMT toolkit for novices. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, Hong Kong, China
  Kreutzer, J., Bastings, J., and Riezler, S.
  
• (2020). Correct me if you can: Learning from error corrections and markings. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translatioin (EAMT), Lisbon, Portugal
  Kreutzer, J., Berger, N., and Riezler, S.
  
• (2020). LibriVoxDeEn: A corpus for German-to-English speech translation and speech recognition. In Proceedings of the Language Resources and Evaluation Conference (LREC), Marseille, France
  Beilharz, B., Sun, X., Karimova, S., and Riezler, S.
  
• (2021). Cascaded models with cyclic feedback for direct speech translation. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, Canada
  Lam, T. K., Schamoni, S., and Riezler, S.
  
• (2021). On-the-fly aligned data augmentation for sequence-to-sequence asr. In Proceedings of the 22th Annual Conference of the International Speech Communication Association (INTERSPEECH), Brno, Czech Republic
  Lam, T. K., Ohta, M., Schamoni, S., and Riezler, S.