Communication systems using neural network-based transceivers with autoencoder-driven end-to-end learning
Final Report Abstract
When this project started in 2018, the idea of incorporating trainable components in communication systems was very promising and a completely new field of research opened up. The vision of end-to-end learned autoencoder-based communication systems was already formulated, however, a practical implementation of such a system that actually surpasses conventional systems in any metric did not yet exist. Thus, the concept of autoencoder-based systems was still in its infancy. In this project, we first focused on the synchronization problems that were one of the major obstacles in previous work. The idea was to solve issues due to timing offsets to be able to also measure the gains we already saw on simulated channels. This was done by embedding the autoencoder’s signaling in a conventional orthogonal frequency division multiplex (OFDM) framework, which solved most of the synchronization problems and reduced channel alterations to single-tap equalization. Another promising approach was to enhance the autoencoder system with an outer FEC code. From this, the idea of using successfully decoded codewords for retraining of an NN-based receiver emerged, which was also patented. In the beginning of this project and according to the objectives of WP1, we also investigated topics in a broader range of the field of deep learning for communications. Among them were investigations on the capabilities of recurrent NNs using the example of decoding convolutional codes, using channel state information (CSI) data which accrue in multiple-input multiple-output (MIMO) systems to infer the position of the transmitter with a NN, and deep learning-based polar code design. One of the key findings of this project was the necessity to switch from a symbol-wise autoencoder that encodes and decodes a message of information to a bit-wise autoencoder that encodes and decodes a vector of bits while also learning a bit-labeling scheme. This change was only possible by developing suitable training strategies and is key to allow the autoencoder system to integrate into a BICM system and to optimally cooperate with outer BMD. In the course of this we also proposed an IDD autoencoder structure that allows to recover even more information from the learned signaling and we proposed to specifically design the outer LDPC code for the learned signaling of the autoencoder as well. Due to these enhancements we were able to observe significant geometrical shaping gains, which can (based on the autoencoder concept) theoretically be achieved over any kind of channel. And while early implementations of an autoencoder communication system were not able to surpass conventional systems, we were now able to measure significantly better BERs with the BICM autoencoder compared to conventional modulation schemes. To the end of the project we tackled the missing channel gradient problem, which is the lack of a gradient through an actual channel that prevents a stochastic gradient descent (SGD)- based training of the transmitter. For this we proposed to learn to mimic the channel using a Wasserstein generative adversarial network (GAN) and then, in a second step, train the autoencoder end-to-end while using the generator of the WGAN as a differentiable channel model. We also investigated a novel Turbo-autoencoder architecture that allows an autoencoder system to encode k >> 16 bits in a single message and thereby circumvents the curse of dimensionality. In brief, the focus of this project was to examine the potential of autoencoder-based communication systems and deep learning for the physical layer in general.
Publications
- “Iterative Detection and Decoding for Trainable Communication Systems”
Jakob Hoydis, Sebastian Cammerer, Sebastian Dörner, Jannis Clausius, Stephan ten Brink
- “Neural Network-based Multi-user Identification and Synchronization Technique”
Jakob Hoydis, Sebastian Cammerer, Sebastian Dörner, Jannis Clausius, Stephan ten Brink
- “Turbo Finetuning for Trainable Communication Systems”
Jakob Hoydis, Sebastian Cammerer, Sebastian Dörner, Stefan Schibisch
- “OFDM-autoencoder for end-to-end learning of communications systems”. In: IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). 2018, pp. 1–5
Alexander Felix, Sebastian Cammerer, Sebastian Dörner, Jakob Hoydis, and Stephan ten Brink
(See online at https://doi.org/10.1109/SPAWC.2018.8445920) - “On Deep Learning-Based Massive MIMO Indoor User Localization”. In: 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). 2018
Maximilian Arnold, Sebastian Dörner, Sebastian Cammerer, and Stephan ten Brink
(See online at https://doi.org/10.1109/SPAWC.2018.8446013) - “Online label recovery for deep learning-based communication through error correcting codes”. In: IEEE 15th International Symposium on Wireless Communication Systems (ISWCS). 2018, pp. 1–5
Stefan Schibisch, Sebastian Cammerer, Sebastian Dörner, Jakob Hoydis, and Stephan ten Brink
(See online at https://doi.org/10.1109/ISWCS.2018.8491189) - “Deep Learning-Based Polar Code Design”. In: 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton). 2019, pp. 177–183
Moustafa Ebada, Sebastian Cammerer, Ahmed Elkelesh, and Stephan ten Brink
(See online at https://doi.org/10.1109/ALLERTON.2019.8919804) - “On Recurrent Neural Networks for Sequence-based Processing in Communications”. In: 2019 53rd Asilomar Conference on Signals, Systems, and Computers. IEEE. 2019, pp. 537–543
Daniel Tandler, Sebastian Dörner, Sebastian Cammerer, and Stephan ten Brink
(See online at https://doi.org/10.1109/IEEECONF44664.2019.9048728) - Trainable Communication Systems, PhD Dissertation, Universität Stuttgart, 2020
Sebastian Cammerer
- “Trainable Communication Systems: Concepts and Prototype”. In: IEEE Transactions on Communications 68.9 (2020), pp. 5489–5503
Sebastian Cammerer, Fayçal Ait Aoudia, Sebastian Dörner, Maximilian Stark, Jakob Hoydis, and Stephan ten Brink
(See online at https://doi.org/10.1109/TCOMM.2020.3002915) - “WGAN-based Autoencoder Training Over-the-air”. In: 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). 2020, pp. 1–5
Sebastian Dörner, Marcus Henninger, Sebastian Cammerer, and Stephan ten Brink
(See online at https://doi.org/10.1109/SPAWC48557.2020.9154335) - “Serial vs. Parallel Turbo-Autoencoders and Accelerated Training for Learned Channel Codes”. In: 2021 11th International Symposium on Topics in Coding (ISTC). 2021, pp. 1–5
Jannis Clausius, Sebastian Dörner, Sebastian Cammerer, and Stephan ten Brink
(See online at https://doi.org/10.1109/ISTC49272.2021.9594130)
