Research

My research interests lie in the intersection of machine learning and optimization, as well as their applications on data mining and biomedical data science.

In particular, my research focuses on answering the following three questions:

Modeling: How to build effective machine learning models to deal with the complex structure of data?
Optimization: How to design efficient optimization algorithms to train machine learning models?
Application: How to apply machine learning models to practical applications?

Modeling: Machine/Deep Learning on Graphs

To have a well-performing machine learning model, a critical step is to capture the intrinsic structure of the data, such as the local correlation of pixels in the image data, the dependence between different words in the language data. Different from these regular data, graph data encode more complicated relational information between different instances. Regular machine learning models fail to deal with the intrinsic relational information so that they cannot be applied to graph data directly. My research focus is to design new machine/deep learning models to effectively explore the relational information for graph data.

Robust Self-Supervised Structural Graph Neural Network for Social Network Prediction.
Yanfu Zhang, Hongchang Gao, Jian Pei, Heng Huang. WWW 2022.

Conditional Random Field Enhanced Graph Convolutional Neural Networks.
Hongchang Gao, Jian Pei, Heng Huang. KDD 2019.

ProGAN: Network Embedding via Proximity Generative Adversarial Network.
Hongchang Gao, Jian Pei, Heng Huang. KDD 2019.

Self-Paced Network Embedding.
Hongchang Gao, Heng Huang. KDD 2018.

Deep Attributed Network Embedding.
Hongchang Gao, Heng Huang. IJCAI 2018.

Local Centroids Structured Non-Negative Matrix Factorization.
Hongchang Gao, Feiping Nie, Heng Huang. AAAI 2017.

Multi-view Subspace Clustering.
Hongchang Gao, Feiping Nie, Xuelong Li, Heng Huang. ICCV 2015.

Optimization: Large-Scale Optimization/Training Methods

After having a machine learning model, a critical step is to optimize this model to get optimal model parameters. Considering the scalability of models and datasets, my research is focusing on designing efficient stochastic optimization algorithms to train large-scale non-convex machine learning models. In addition, deep neural networks are highly non-linear. They are easy to overfit and difficult to train. Besides optimization methods, I also work on designing efficient training methods for deep neural networks to avoid the overfitting issue and stabilize the training procedure.

Federated Stochastic Bilevel Optimization with Fully First-Order Gradients.
Yihan Zhang#, Rohit Dhaipule, Chiu C. Tan, Haibin Ling, Hongchang Gao. IJCAI 2025.

A Federated Stochastic Multi-level Compositional Minimax Algorithm for Deep AUC Maximization.
Xinwen Zhang#, Ali Payani, Myungjin Lee, Richard Souvenir, Hongchang Gao. ICML 2024.

A Doubly Recursive Stochastic Compositional Gradient Descent Method for Federated Multi-Level Compositional Optimization.
Hongchang Gao. ICML 2024.

Decentralized Multi-Level Compositional Optimization Algorithms with Level-Independent Convergence Rate.
Hongchang Gao. AISTATS 2024.

Federated Compositional Deep AUC Maximization.
Xinwen Zhang*#, Yihan Zhang*#, Tianbao Yang, Richard Souvenir, Hongchang Gao. NeurIPS 2023.

Communication-Efficient Stochastic Gradient Descent Ascent with Momentum Algorithms.
Yihan Zhang#, Meikang Qiu, Hongchang Gao. IJCAI 2023.

On the Convergence of Distributed Stochastic Bilevel Optimization Algorithms over a Network.
Hongchang Gao, Bin Gu, My T. Thai. AISTATS 2023.

On the Convergence of Local Stochastic Compositional Gradient Descent with Momentum.
Hongchang Gao, Junyi Li, Heng Huang. ICML2022.

Gradient-Free Method for Heavily Constrained Nonconvex Optimization.
Wanli Shi, Hongchang Gao, Bin Gu. ICML2022.

Efficient Decentralized Stochastic Gradient Descent Method for Nonconvex Finite-Sum Optimization Problems.
Wenkang Zhan, Gang Wu, Hongchang Gao. AAAI 2022.

Fast Training Method for Stochastic Compositional Optimization Problems.
Hongchang Gao, Heng Huang. NeurIPS 2021.

Sample Efficient Decentralized Stochastic Frank-Wolfe Methods for Continuous DR-Submodular Maximization.
Hongchang Gao, Hanzi Xu, Slobodan Vucetic. IJCAI 2021.

On the Convergence of Stochastic Compositional Gradient Descent Ascent Method.
Hongchang Gao, Xiaoqian Wang, Lei Luo, Mindy Shi. IJCAI 2021.

On the Convergence of Communication-Efficient Local SGD for Federated Learning.
Hongchang Gao, An Xu, Heng Huang. AAAI 2021.

Provable Distributed Stochastic Gradient Descent with Delayed Updates.
Hongchang Gao, Gang Wu, Ryan Rossi. SDM 2021.

Faster Stochastic Second Order Method for Large-scale Machine Learning Models.
Hongchang Gao, Heng Huang. SDM 2021.

Can Stochastic Zeroth-Order Frank-Wolfe Method Converge Faster for Non-Convex Problems?
Hongchang Gao, Heng Huang. ICML 2020.

Demystifying Dropout.
Hongchang Gao, Jian Pei, Heng Huang. ICML 2019.

Stochastic Second-Order Method for Large-Scale Nonconvex Sparse Learning Models.
Hongchang Gao, Heng Huang. IJCAI 2018.

Application: Biomedical Data Science & Online Advertising

Besides the methodology side, I am also interested in applying machine learning to other fields, such as bioinformatics, online advertising. To design effective machine learning models for practical applications, it is important to incorporate the domain-specific knowledge. To bridge the gap between general machine learning models and the domain-specific application, my research work is to design new machine/deep learning models to fully exploit domain knowledge for better prediction.

New Robust Clustering Model for Identifying Cancer Genome Landscapes.
Hongchang Gao*, Xiaoqian Wang*, Heng Huang. ICDM 2016.

Anatomical Annotations for Drosophila Gene Expression Patterns via Multi-Dimensional Visual Descriptors Integration: Multi-Dimensional Feature Learning.
Hongchang Gao, Lin Yan, Weidong Cai, Heng Huang. KDD 2015.

Identifying Connectome Module Patterns via New Balanced Multi-graph Normalized Cut.
Hongchang Gao, Chengtao Cai, Jingwen Yan, Lin Yan, Joaquín Goñi Cortes, Yang Wang, Feiping Nie, John D. West, Andrew J. Saykin, Li Shen, Heng Huang. MICCAI 2015.

Attention Convolutional Neural Network for Advertiser-level Click-through Rate Forecasting.
Hongchang Gao, Deguang Kong, Miao Lu, Xiao Bai, Jian Yang. WWW 2018.