Wireless for Machine Learning

Dr. Carlo Fischione, carlofi@kth.se
KTH Royal Institute of Technology, Sweden

Carlo Fischione is Professor at KTH Royal Institute of Technology, Sweden. He is the Chair of the IEEE Machine Learning for Communications Emerging Technological Initiative, and Director of the “Data Science” Micro Degree Program of KTH Royal Institute of Technology, Sweden, dedicated to Ericsson’s researchers worldwide. He received the Ph.D. degree in Electrical and Information Engineering in 2005 and the Laurea degree in Electronic Engineering (Summa cum Laude) in 2001 from University of L’Aquila, Italy. He has had faculty positions at the University of California at Berkley, MIT Massachusetts Institute of Technology, and Harvard University. He was recipient of numerous awards, including the Best Paper Awards from the IEEE Transactions on Communications (2018), the IEEE Transactions on Industrial Informatics (2007), and several Best Paper Awards at IEEE conferences. He has co-authored over 200 publications, including book, book chapters, journals, conferences, and patents. He has offered consultancy to numerous technology companies such as ABB Corporate Research, Berkeley Wireless Sensor Network Lab, Ericsson Research, Synopsys, and United Technology Research Center. His research interests include optimization with applications to networks, wireless and sensor networks, and Internet of Things. He is Editor of the IEEE Transactions on Communications and the IEEE Journal on Selected Areas in Communications series Machine Learning in Communications and Networks.

Dr. Viktoria Fodor, vjfodor@kth.se
KTH Royal Institute of Technology, Sweden

Viktoria Fodor is Professor of Communication Networks at KTH Royal Institute of Technology, Sweden. She received the M.Sc. and Ph.D. degrees from the Budapest University of Technology and Economics, Budapest, Hungary, in 1992 and 1999, respectively, in computer engineering. She received habilitation qualification (docent) from KTH in 2011. In 1998, she was a senior researcher with the Hungarian Telecommunication Company. Since 1999, she has been with KTH. Her research interests include networks and distributed systems, stochastic modeling, protocol design, edge computing, and machine learning over networks. She has published over 100 scientific publications, is Associate Editor of IEEE Transactions of Network and Service Management and Wiley Transactions on Emerging Telecommunications Technologies and Area Chair of IEEE Infocom 2019.

Dr. José Mairton B. da Silva Jr., jmbdsj@kth.se
KTH Royal Institute of Technology, Sweden

José Mairton B. da Silva Jr. received his Ph.D. degree in Electrical Engineering and Computer Science from KTH Royal Institute of Technology, Sweden, in 2019. He received his BSc (honors) and MSc degree in Telecommunications Engineering from the Federal University of Ceará, Brazil, in 2012 and 2014, respectively. He was a research engineer at the Wireless Telecommunication Research Group (GTEL), Brazil, from 2012 to 2015. During 2018, he was a visiting researcher at Rice University, USA. He is currently a Postdoctoral Researcher at KTH, and Secretary for the IEEE ComSoc Emerging Technology Initiative on Full-Duplex Communications. His research interests include distributed machine learning and optimization over networks.

Motivation

When we study machine learning and wireless, we have essentially three research areas:

1. Distributed Machine Learning over Wireless Networks, such as Federated Learning;
2. Machine Learning to redesign Wireless Protocols for any communication service;
3. Fundamentally new Wireless Communication Protocols to support future Machine Learning services.

While 1. and 2. have been the subject of several tutorials at IEEE conference, there are no tutorials on 3. Machine learning has originally been conceived in centralized settings where all data is available. Accordingly, its application on distributed datasets and computations over wireless networks generates new challenges. Even in data centres, it is often reported that communication forms the bottleneck for distributed machine learning despite using high-speed wired connections. The machine learning over wireless challenges stems from communication latency, bandwidth consumption, energy limitations, privacy, security, and wireless phenomena such as path loss, interference, and fading.

Due to the distributed nature of the data sets and computational units in wireless networks, state-of-the-art ML algorithms are heavily dependent on the underlying wireless communication protocols. The result is that current and upcoming wireless networks may be greatly stressed when trying to support state-of-the-art ML algorithms. Data or computation information for ML tasks such as distributed training and inference, especially in dynamic set-up where these computations haveto be re-executed frequently, do not need to be transmitted according to the traditional wireless protocols that seek to minimize interference or avoid simultaneous transmissions in time and frequency. An alternativeapproach guides communication protocoldesign rather than modifying the learning algorithm, creating customized wireless solutions for carrying data needed for the machine learning tasks. Unlike traditional wireless protocol design, the objective of communication for machine learning is not todeliver bits as efficiently as possible, but to distil the intelligence carried within the data. Traditional communication protocols that are designed to maximize data rate and minimize bit errors have been shown to be greatly inefficient for carrying machine learning related data.Instead, wireless for machine learning offers new methods that are better aligned with the learning objectives and invites us to rethink how wireless communication protocols are designed. This idea is illustrated by the prominent example of distributed ML, among many, of Federated Learning, whose computations can be performed “over the air” by wireless communication protocols.This new approach fits well into intelligent edge networks, which has seen a surge of interest from academia and industry alike and is predicted to be a central component of beyond-5G or 6G cellular networks.This suggests that we need to reinvestigate fundamentally new wireless communication protocols (e.g., signal processing methods for physical layer and medium access controls) capable to support distributed ML services. Moreover, an even more ambitious goal, is to design ML and wireless methods in symbiosis.

Structure and content

We divide the tutorial into 6 parts, which are detailed below:

1. Introduction [Presenter: Carlo Fischione]
2. Primer on Distributed Machine Learning [Presenter: Carlo Fischione]
   1. Problem Formulation in Centralized and Distributed Machine Learning
   2. Federated Learning and Other Distributed Learning Methods
3. Primer on Computation over Multiple Access Channels [Presenter: Viktoria Fodor]
   1. Power Modulation Example
4. Computation over Multiple Access Channels for Machine Learning [Presenter: Carlo Fischione]
   1. SISO Networks: Broadband Analog Aggregation, Gradient Sparsification, Federated Distillation, Training with Noisy Gradients, and Digital Aggregation
   2. MIMO Networks: Blind Learning, Cell-free massive MIMO, Beamforming and User Selection co-design
5. Orthogonal Machine Learning-Driven Communications [Presenter: Carlo Fischione]
   1. Importance-Aware Communications: Centralized and DistributedLearning
   2. Radio Resource Management for DistributedLearning: Energy Efficiency, Packet Error Impact, Total Time Budget, Batch Size Selection, Importance Radio Resource Management
6. Open Challenges and Future Research Directions [Presenters: José Mairton B. da Silva Jr and Henrik Hellström]
   1. Channel State Information
   2. Impact of Channel Uncertainty
   3. Analog Transmission for Machine Learning
   4. Security
   5. Implementation in Cellular Networks
   6. Data-Importance Metrics and Staleness