IJCAI 2025 Tutorial: Federated Compositional and Bilevel Optimization

Federated Learning has attracted significant attention in recent years, resulting in the development of numerous methods. However, most of these methods focus solely on traditional minimization problems and fail to address new learning paradigms in machine learning. Therefore, this tutorial focuses on the learning paradigm that can be formulated as the stochastic compositional optimization (SCO) problem and the stochastic bilevel optimization (SBO) problem, as they cover a wide variety of machine learning models beyond traditional minimization problem, such as model-agnostic meta-learning, imbalanced data classification models, contrastive self-supervised learning models, graph neural networks, neural architecture search, etc. The compositional structure and bilevel structures bring unique challenges in computation and communication for federated learning. To address these challenges, a series of federated compositional optimization and federated bilevel optimization methods have been developed in the past few years. However, these advances have not been widely disseminated. Thus, this tutorial aims to introduce the unique challenges, recent advances, and practical applications of federated SCO and SBO. The audience will benefit from this tutorial by gaining a deeper understanding of federated SCO and SBO algorithms and learning how to apply them to real-world applications.

• Section I: Section I: Introduction (15 min).
  

• Section II: Federated Compositional Optimization (35 min).
  

• Section III: Federated Bilevel Optimization (35 min).
  

• Section IV: Summary and Future Directions (10 min).