Dr. Carlo Fischione
KTH Royal Institute of Technology, Sweden
Dr. José Mairton B. da Silva Jr.
KTH Royal Institute of Technology, Sweden
When we study machine learning and wireless, we have essentially three research areas:
- Distributed Machine Learning over Wireless Networks, such as Federated Learning;
- Machine Learning to redesign Wireless Protocols for any communication service;
- Fundamentally new Wireless Communication Protocols to support future Machine Learning services.
While 1. and 2. have been the subject of several tutorials at IEEE conference, there are no tutorials on 3. Machine learning has originally been conceived in centralized settings where all data is available. Accordingly, its application on distributed datasets and computations over wireless networks generates new challenges. Even in data centres, it is often reported that communication forms the bottleneck for distributed machine learning despite using high-speed wired connections. The machine learning over wireless challenges stems from communication latency, bandwidth consumption, energy limitations, privacy, security, and wireless phenomena such as path loss, interference, and fading.
Due to the distributed nature of the data sets and computational units in wireless networks, state-of-the-art ML algorithms are heavily dependent on the underlying wireless communication protocols. The result is that current and upcoming wireless networks may be greatly stressed when trying to support state-of-the-art ML algorithms. Data or computation information for ML tasks such as distributed training and inference, especially in dynamic set-up where these computations haveto be re-executed frequently, do not need to be transmitted according to the traditional wireless protocols that seek to minimize interference or avoid simultaneous transmissions in time and frequency. An alternativeapproach guides communication protocoldesign rather than modifying the learning algorithm, creating customized wireless solutions for carrying data needed for the machine learning tasks. Unlike traditional wireless protocol design, the objective of communication for machine learning is not todeliver bits as efficiently as possible, but to distil the intelligence carried within the data. Traditional communication protocols that are designed to maximize data rate and minimize bit errors have been shown to be greatly inefficient for carrying machine learning related data.Instead, wireless for machine learning offers new methods that are better aligned with the learning objectives and invites us to rethink how wireless communication protocols are designed. This idea is illustrated by the prominent example of distributed ML, among many, of Federated Learning, whose computations can be performed “over the air” by wireless communication protocols.This new approach fits well into intelligent edge networks, which has seen a surge of interest from academia and industry alike and is predicted to be a central component of beyond-5G or 6G cellular networks.This suggests that we need to reinvestigate fundamentally new wireless communication protocols (e.g., signal processing methods for physical layer and medium access controls) capable to support distributed ML services. Moreover, an even more ambitious goal, is to design ML and wireless methods in symbiosis.
Structure and content
We divide the tutorial into 6 parts, which are detailed below:
- Introduction [Presenter: Carlo Fischione]
- Primer on Distributed Machine Learning [Presenter: Carlo Fischione]
- Problem Formulation in Centralized and Distributed Machine Learning
- Federated Learning and Other Distributed Learning Methods
- Primer on Computation over Multiple Access Channels [Presenter: Viktoria Fodor]
- Power Modulation Example
- Computation over Multiple Access Channels for Machine Learning [Presenter: Carlo Fischione]
- SISO Networks: Broadband Analog Aggregation, Gradient Sparsification, Federated Distillation, Training with Noisy Gradients, and Digital Aggregation
- MIMO Networks: Blind Learning, Cell-free massive MIMO, Beamforming and User Selection co-design
- Orthogonal Machine Learning-Driven Communications [Presenter: Carlo Fischione]
- Importance-Aware Communications: Centralized and DistributedLearning
- Radio Resource Management for DistributedLearning: Energy Efficiency, Packet Error Impact, Total Time Budget, Batch Size Selection, Importance Radio Resource Management
- Open Challenges and Future Research Directions [Presenters: José Mairton B. da Silva Jr and Henrik Hellström]
- Channel State Information
- Impact of Channel Uncertainty
- Analog Transmission for Machine Learning
- Implementation in Cellular Networks
- Data-Importance Metrics and Staleness